2024-06-03T00:42:41.729 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-03T00:42:41.729 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} --name teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748 -- /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/orig.config.yaml 2024-06-03T00:42:41.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T00:42:41.875 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7738748' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2064 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 125/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHso5spWIqRP7mBaZigJSXTW5L8bVnV5Us+bZjG9spd1XWF5tssWpcfJFUumVWb04r2ijBDE/aVAvQHC0RfARw= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeZGzvJ6b1i1bwU+sJyR4auKgGZ7784i83qOP4hsn1pDqJcGZcREeWU/A1lXhT4FGiwqu7S1KiOb930UyKOonw= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnTi/9ntMDMcOh4b57JmPVQBkTTegQfwyj0jVjjvTcXzhlTgQdBp7cm65xU2aq7LDEIjmY3BfvaxKvMPaYdoWA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-02_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-03T00:42:41.875 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa; will attempt to use it 2024-06-03T00:42:41.876 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks 2024-06-03T00:42:41.876 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-03T00:42:41.878 INFO:teuthology.task.internal:Checking packages... 2024-06-03T00:42:41.900 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-03T00:42:41.900 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-03T00:42:41.900 INFO:teuthology.packaging:ref: None 2024-06-03T00:42:41.900 INFO:teuthology.packaging:tag: None 2024-06-03T00:42:41.900 INFO:teuthology.packaging:branch: squid 2024-06-03T00:42:41.900 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:42:41.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-03T00:42:42.117 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2952.gdfcc4b3c 2024-06-03T00:42:42.118 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-03T00:42:42.125 INFO:teuthology.task.internal:no buildpackages task found 2024-06-03T00:42:42.126 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-03T00:42:42.266 INFO:teuthology.task.internal:Saving configuration 2024-06-03T00:42:42.290 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-03T00:42:42.297 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-03T00:42:42.323 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 00:35:49.332793', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHso5spWIqRP7mBaZigJSXTW5L8bVnV5Us+bZjG9spd1XWF5tssWpcfJFUumVWb04r2ijBDE/aVAvQHC0RfARw='} 2024-06-03T00:42:42.344 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 00:35:49.335013', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeZGzvJ6b1i1bwU+sJyR4auKgGZ7784i83qOP4hsn1pDqJcGZcREeWU/A1lXhT4FGiwqu7S1KiOb930UyKOonw='} 2024-06-03T00:42:42.369 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-03 00:35:49.334013', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnTi/9ntMDMcOh4b57JmPVQBkTTegQfwyj0jVjjvTcXzhlTgQdBp7cm65xU2aq7LDEIjmY3BfvaxKvMPaYdoWA='} 2024-06-03T00:42:42.369 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-03T00:42:42.375 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-03T00:42:42.375 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-03T00:42:42.376 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-03T00:42:42.376 INFO:teuthology.run_tasks:Running task console_log... 2024-06-03T00:42:42.455 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f387d8fbbe0>, signals=[15]) 2024-06-03T00:42:42.455 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-03T00:42:42.461 INFO:teuthology.task.internal:Opening connections... 2024-06-03T00:42:42.461 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T00:42:42.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:42:42.539 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-06-03T00:42:42.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:42:42.613 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-06-03T00:42:42.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:42:42.688 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-03T00:42:42.695 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-06-03T00:42:42.712 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-06-03T00:42:42.713 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-06-03T00:42:42.769 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-06-03T00:42:42.769 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-06-03T00:42:42.769 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-06-03T00:42:42.769 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-06-03T00:42:42.769 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T00:42:42.770 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T00:42:42.771 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-06-03T00:42:42.792 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-06-03T00:42:42.810 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-06-03T00:42:42.811 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-06-03T00:42:42.867 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T00:42:42.868 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T00:42:42.869 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T00:42:42.869 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-06-03T00:42:42.892 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-06-03T00:42:42.910 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-06-03T00:42:42.911 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-06-03T00:42:42.968 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-06-03T00:42:42.968 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-06-03T00:42:42.968 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-06-03T00:42:42.969 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-06-03T00:42:42.969 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-06-03T00:42:42.969 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-06-03T00:42:42.969 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-03T00:42:42.969 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-06-03T00:42:42.969 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-06-03T00:42:42.970 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-03T00:42:42.970 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-06-03T00:42:42.970 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-03T00:42:42.970 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-03T00:42:42.970 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-03T00:42:42.970 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-06-03T00:42:42.992 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-03T00:42:43.002 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-03T00:42:43.007 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-03T00:42:43.008 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-06-03T00:42:43.010 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-06-03T00:42:43.012 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-06-03T00:42:43.028 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-03T00:42:43.034 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-03T00:42:43.034 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-06-03T00:42:43.067 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-06-03T00:42:43.069 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-06-03T00:42:43.104 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-03T00:42:43.167 INFO:teuthology.run_tasks:Running task kernel... 2024-06-03T00:42:43.182 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-03T00:42:43.183 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-06-03T00:42:43.183 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-06-03T00:42:43.183 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T00:42:43.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T00:42:43.200 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-06-03T00:42:43.471 DEBUG:teuthology.task.kernel:sha1 for client.0 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T00:42:43.471 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-06-03T00:42:43.471 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-06-03T00:42:43.488 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-437.el9.x86_64 2024-06-03T00:42:43.488 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T00:42:43.489 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-06-03T00:42:43.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum upgrade -y linux-firmware 2024-06-03T00:42:46.480 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.1 MB 00:01 2024-06-03T00:42:48.240 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-06-03T00:42:52.902 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-06-03T00:42:54.717 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 17 kB 00:00 2024-06-03T00:42:55.860 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-06-03T00:43:00.977 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-06-03T00:43:03.111 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-06-03T00:43:03.112 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-06-03T00:43:03.112 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-06-03T00:43:03.224 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-03T00:43:03.225 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-03T00:43:03.456 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-06-03T00:43:03.456 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-06-03T00:43:03.458 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-06-03T00:43:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03 00:43:17 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-06-03T00:43:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:FINISHED --2024-06-03 00:43:17-- 2024-06-03T00:43:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:Total wall clock time: 14s 2024-06-03T00:43:17.788 INFO:teuthology.orchestra.run.smithi094.stderr:Downloaded: 1 files, 340M in 14s (24.2 MB/s) 2024-06-03T00:43:17.791 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-03T00:43:17.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-06-03T00:43:19.862 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying... ######################################## 2024-06-03T00:43:19.997 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing... ######################################## 2024-06-03T00:43:20.115 INFO:teuthology.orchestra.run.smithi094.stdout:Updating / installing... 2024-06-03T00:43:28.022 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-06-03T00:44:07.977 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-06-03T00:44:07.977 DEBUG:teuthology.orchestra.run.smithi094:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-06-03T00:44:08.058 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/config-6.9.0-gb9459a26b088 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-06-03T00:44:08.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-06-03T00:44:08.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-06-03T00:44:08.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-06-03T00:44:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-06-03T00:44:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-06-03T00:44:08.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-06-03T00:44:08.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-06-03T00:44:08.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-06-03T00:44:08.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-06-03T00:44:08.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-06-03T00:44:08.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-06-03T00:44:08.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-06-03T00:44:08.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-06-03T00:44:08.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-06-03T00:44:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-06-03T00:44:08.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-06-03T00:44:08.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-06-03T00:44:08.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-06-03T00:44:08.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-06-03T00:44:08.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-06-03T00:44:08.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-06-03T00:44:08.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-06-03T00:44:08.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-06-03T00:44:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-06-03T00:44:08.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-06-03T00:44:08.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-06-03T00:44:08.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-06-03T00:44:08.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-06-03T00:44:08.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-06-03T00:44:08.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-06-03T00:44:08.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-06-03T00:44:08.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-06-03T00:44:08.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-06-03T00:44:08.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-06-03T00:44:08.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-06-03T00:44:08.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-06-03T00:44:08.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-06-03T00:44:08.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-06-03T00:44:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-06-03T00:44:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-06-03T00:44:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-06-03T00:44:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-06-03T00:44:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-06-03T00:44:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-06-03T00:44:08.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-06-03T00:44:08.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-06-03T00:44:08.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-06-03T00:44:08.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-06-03T00:44:08.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-06-03T00:44:08.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-06-03T00:44:08.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-06-03T00:44:08.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-06-03T00:44:08.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-06-03T00:44:08.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-06-03T00:44:08.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-06-03T00:44:08.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-06-03T00:44:08.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-06-03T00:44:08.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-06-03T00:44:08.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-06-03T00:44:08.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-06-03T00:44:08.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-06-03T00:44:08.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-06-03T00:44:08.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-06-03T00:44:08.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-06-03T00:44:08.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-06-03T00:44:08.108 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-06-03T00:44:08.108 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-06-03T00:44:08.108 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-06-03T00:44:08.108 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-06-03T00:44:08.108 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-06-03T00:44:08.108 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-06-03T00:44:08.108 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-06-03T00:44:08.109 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-06-03T00:44:08.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-06-03T00:44:08.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-06-03T00:44:08.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-06-03T00:44:08.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-06-03T00:44:08.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-06-03T00:44:08.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-06-03T00:44:08.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-06-03T00:44:08.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-06-03T00:44:08.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-06-03T00:44:08.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-06-03T00:44:08.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-06-03T00:44:08.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-06-03T00:44:08.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-06-03T00:44:08.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-06-03T00:44:08.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-06-03T00:44:08.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-06-03T00:44:08.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-06-03T00:44:08.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-06-03T00:44:08.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-06-03T00:44:08.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-06-03T00:44:08.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-06-03T00:44:08.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-06-03T00:44:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-06-03T00:44:08.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-06-03T00:44:08.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-06-03T00:44:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-06-03T00:44:08.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-06-03T00:44:08.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-06-03T00:44:08.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-06-03T00:44:08.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-06-03T00:44:08.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-06-03T00:44:08.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-06-03T00:44:08.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-06-03T00:44:08.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-06-03T00:44:08.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-06-03T00:44:08.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-06-03T00:44:08.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-06-03T00:44:08.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-06-03T00:44:08.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-06-03T00:44:08.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-06-03T00:44:08.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-06-03T00:44:08.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-06-03T00:44:08.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-06-03T00:44:08.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-06-03T00:44:08.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-06-03T00:44:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-06-03T00:44:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-06-03T00:44:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-06-03T00:44:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-06-03T00:44:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-06-03T00:44:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-06-03T00:44:08.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-06-03T00:44:08.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-06-03T00:44:08.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-06-03T00:44:08.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-06-03T00:44:08.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-06-03T00:44:08.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-06-03T00:44:08.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-06-03T00:44:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-06-03T00:44:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-06-03T00:44:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-06-03T00:44:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-06-03T00:44:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-06-03T00:44:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-06-03T00:44:08.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-06-03T00:44:08.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-06-03T00:44:08.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-06-03T00:44:08.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-06-03T00:44:08.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-06-03T00:44:08.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-06-03T00:44:08.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-06-03T00:44:08.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-06-03T00:44:08.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-06-03T00:44:08.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-06-03T00:44:08.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-06-03T00:44:08.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-06-03T00:44:08.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-06-03T00:44:08.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-06-03T00:44:08.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-06-03T00:44:08.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-06-03T00:44:08.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-06-03T00:44:08.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-06-03T00:44:08.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-06-03T00:44:08.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-06-03T00:44:08.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-06-03T00:44:08.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-06-03T00:44:08.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-06-03T00:44:08.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-06-03T00:44:08.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-06-03T00:44:08.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-06-03T00:44:08.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-06-03T00:44:08.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-06-03T00:44:08.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-06-03T00:44:08.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-06-03T00:44:08.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-06-03T00:44:08.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-06-03T00:44:08.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-06-03T00:44:08.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-06-03T00:44:08.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-06-03T00:44:08.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-06-03T00:44:08.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-06-03T00:44:08.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-06-03T00:44:08.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-06-03T00:44:08.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-06-03T00:44:08.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-06-03T00:44:08.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-06-03T00:44:08.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-06-03T00:44:08.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-06-03T00:44:08.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-06-03T00:44:08.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-06-03T00:44:08.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-06-03T00:44:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-06-03T00:44:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-06-03T00:44:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-06-03T00:44:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-06-03T00:44:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-06-03T00:44:08.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-06-03T00:44:08.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-06-03T00:44:08.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-06-03T00:44:08.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-06-03T00:44:08.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-06-03T00:44:08.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-06-03T00:44:08.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-06-03T00:44:08.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-06-03T00:44:08.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-06-03T00:44:08.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-06-03T00:44:08.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-06-03T00:44:08.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-06-03T00:44:08.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-06-03T00:44:08.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-06-03T00:44:08.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-06-03T00:44:08.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-06-03T00:44:08.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-06-03T00:44:08.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-06-03T00:44:08.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-06-03T00:44:08.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-06-03T00:44:08.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-06-03T00:44:08.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-06-03T00:44:08.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-06-03T00:44:08.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-06-03T00:44:08.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-06-03T00:44:08.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-06-03T00:44:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-06-03T00:44:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-06-03T00:44:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-06-03T00:44:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-06-03T00:44:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-06-03T00:44:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-06-03T00:44:08.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-06-03T00:44:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-06-03T00:44:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-06-03T00:44:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-06-03T00:44:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-06-03T00:44:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-06-03T00:44:08.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-06-03T00:44:08.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-06-03T00:44:08.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-06-03T00:44:08.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-06-03T00:44:08.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-06-03T00:44:08.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-06-03T00:44:08.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-06-03T00:44:08.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-06-03T00:44:08.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-06-03T00:44:08.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-06-03T00:44:08.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-06-03T00:44:08.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-06-03T00:44:08.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-06-03T00:44:08.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-06-03T00:44:08.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-06-03T00:44:08.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-06-03T00:44:08.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-06-03T00:44:08.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-06-03T00:44:08.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-06-03T00:44:08.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-06-03T00:44:08.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-06-03T00:44:08.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-06-03T00:44:08.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-06-03T00:44:08.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-06-03T00:44:08.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-06-03T00:44:08.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-06-03T00:44:08.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-06-03T00:44:08.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-06-03T00:44:08.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-06-03T00:44:08.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-06-03T00:44:08.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-06-03T00:44:08.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-06-03T00:44:08.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-06-03T00:44:08.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-06-03T00:44:08.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-06-03T00:44:08.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-06-03T00:44:08.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-06-03T00:44:08.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-06-03T00:44:08.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-06-03T00:44:08.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-06-03T00:44:08.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-06-03T00:44:08.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-06-03T00:44:08.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-06-03T00:44:08.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-06-03T00:44:08.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-06-03T00:44:08.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-06-03T00:44:08.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-06-03T00:44:08.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-06-03T00:44:08.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-06-03T00:44:08.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-06-03T00:44:08.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-06-03T00:44:08.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-06-03T00:44:08.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-06-03T00:44:08.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-06-03T00:44:08.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-06-03T00:44:08.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-06-03T00:44:08.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-06-03T00:44:08.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-06-03T00:44:08.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-06-03T00:44:08.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-06-03T00:44:08.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-06-03T00:44:08.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-06-03T00:44:08.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-06-03T00:44:08.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-06-03T00:44:08.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-06-03T00:44:08.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-06-03T00:44:08.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-06-03T00:44:08.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-06-03T00:44:08.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-06-03T00:44:08.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-06-03T00:44:08.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-06-03T00:44:08.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-06-03T00:44:08.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-06-03T00:44:08.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-06-03T00:44:08.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-06-03T00:44:08.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-06-03T00:44:08.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-06-03T00:44:08.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-06-03T00:44:08.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-06-03T00:44:08.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-06-03T00:44:08.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-06-03T00:44:08.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-06-03T00:44:08.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-06-03T00:44:08.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-06-03T00:44:08.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-06-03T00:44:08.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-06-03T00:44:08.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-06-03T00:44:08.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-06-03T00:44:08.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-06-03T00:44:08.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-06-03T00:44:08.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-06-03T00:44:08.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-06-03T00:44:08.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-06-03T00:44:08.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-06-03T00:44:08.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-06-03T00:44:08.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-06-03T00:44:08.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-06-03T00:44:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-06-03T00:44:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-06-03T00:44:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-06-03T00:44:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-06-03T00:44:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-06-03T00:44:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-06-03T00:44:08.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-06-03T00:44:08.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-06-03T00:44:08.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-06-03T00:44:08.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-06-03T00:44:08.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-06-03T00:44:08.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-06-03T00:44:08.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-06-03T00:44:08.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-06-03T00:44:08.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-06-03T00:44:08.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-06-03T00:44:08.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-06-03T00:44:08.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-06-03T00:44:08.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-06-03T00:44:08.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-06-03T00:44:08.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-06-03T00:44:08.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-06-03T00:44:08.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-06-03T00:44:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-06-03T00:44:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-06-03T00:44:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-06-03T00:44:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-06-03T00:44:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-06-03T00:44:08.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-06-03T00:44:08.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-06-03T00:44:08.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-06-03T00:44:08.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-06-03T00:44:08.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-06-03T00:44:08.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-06-03T00:44:08.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-06-03T00:44:08.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-06-03T00:44:08.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-06-03T00:44:08.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-06-03T00:44:08.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-06-03T00:44:08.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-06-03T00:44:08.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-06-03T00:44:08.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-06-03T00:44:08.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-06-03T00:44:08.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-06-03T00:44:08.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-06-03T00:44:08.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-06-03T00:44:08.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-06-03T00:44:08.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-06-03T00:44:08.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-06-03T00:44:08.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-06-03T00:44:08.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-06-03T00:44:08.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-06-03T00:44:08.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-06-03T00:44:08.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-06-03T00:44:08.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-06-03T00:44:08.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-06-03T00:44:08.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-06-03T00:44:08.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-06-03T00:44:08.185 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-06-03T00:44:08.186 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-06-03T00:44:08.187 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-06-03T00:44:08.188 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-06-03T00:44:08.189 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-06-03T00:44:08.190 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-06-03T00:44:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-06-03T00:44:08.192 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-06-03T00:44:08.193 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-06-03T00:44:08.194 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-06-03T00:44:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-06-03T00:44:08.196 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-06-03T00:44:08.197 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-06-03T00:44:08.198 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-06-03T00:44:08.199 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-06-03T00:44:08.200 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-06-03T00:44:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-06-03T00:44:08.202 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-06-03T00:44:08.203 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-06-03T00:44:08.204 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-06-03T00:44:08.205 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-06-03T00:44:08.206 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-06-03T00:44:08.207 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-06-03T00:44:08.208 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-06-03T00:44:08.209 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-06-03T00:44:08.210 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-06-03T00:44:08.211 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-06-03T00:44:08.212 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-06-03T00:44:08.213 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-06-03T00:44:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-06-03T00:44:08.215 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-06-03T00:44:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-06-03T00:44:08.217 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-06-03T00:44:08.218 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-06-03T00:44:08.219 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-06-03T00:44:08.220 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-06-03T00:44:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-06-03T00:44:08.222 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-06-03T00:44:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-06-03T00:44:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-06-03T00:44:08.225 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-06-03T00:44:08.226 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-06-03T00:44:08.227 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-06-03T00:44:08.228 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-06-03T00:44:08.229 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-06-03T00:44:08.230 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-06-03T00:44:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-06-03T00:44:08.232 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-06-03T00:44:08.233 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-06-03T00:44:08.234 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-06-03T00:44:08.235 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-03T00:44:08.240 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-06-03T00:44:08.240 DEBUG:teuthology.orchestra.run.smithi094:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout:postinstall scriptlet (using /bin/sh): 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout:for file in vmlinuz System.map config; do 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout: fi 2024-06-03T00:44:08.263 INFO:teuthology.orchestra.run.smithi094.stdout:done 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:preuninstall scriptlet (using /bin/sh): 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-06-03T00:44:08.264 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-06-03T00:44:08.265 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-06-03T00:44:08.341 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-03T00:44:08.342 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-06-03T00:44:08.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:44:08.343 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-06-03T00:44:08.343 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-03T00:44:08.344 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 6.9.0-gb9459a26b088 2024-06-03T00:44:08.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-03T00:44:08.950 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-06-03T00:44:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-03T00:44:10.350 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-06-03T00:44:10.352 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-06-03T00:44:10.372 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.rcB6r4RdAd 2024-06-03T00:44:10.373 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rcB6r4RdAd 2024-06-03T00:44:10.438 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.rcB6r4RdAd 2024-06-03T00:44:10.575 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.rcB6r4RdAd is 6KB 2024-06-03T00:44:10.622 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.rcB6r4RdAd 2024-06-03T00:44:10.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-06-03T00:44:10.697 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-03T00:44:10.698 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-03T00:44:10.698 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-03T00:44:10.698 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-06-03T00:44:10.699 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-06-03T00:44:10.862 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-06-03T00:44:40.864 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-03T00:44:40.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-06-03T00:44:40.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:45:40.869 DEBUG:teuthology.orchestra.remote:timed out 2024-06-03T00:45:49.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-06-03T00:45:49.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:45:57.498 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-06-03T00:45:58.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-06-03T00:45:58.266 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gb9459a26b088"... 2024-06-03T00:45:58.266 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-06-03T00:45:58.302 INFO:teuthology.orchestra.run.smithi094.stdout:6.9.0-gb9459a26b088 2024-06-03T00:45:58.302 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-06-03T00:45:58.302 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-03T00:45:58.303 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-03T00:45:59.305 DEBUG:teuthology.parallel:result is None 2024-06-03T00:45:59.306 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-03T00:45:59.317 INFO:teuthology.task.internal:Creating test directory... 2024-06-03T00:45:59.318 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T00:45:59.323 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T00:45:59.326 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-03T00:45:59.367 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-03T00:45:59.389 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-03T00:45:59.447 INFO:teuthology.task.internal:Creating archive directory... 2024-06-03T00:45:59.447 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T00:45:59.452 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T00:45:59.454 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-03T00:45:59.552 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-03T00:45:59.559 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-03T00:45:59.559 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T00:45:59.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T00:45:59.576 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-03T00:45:59.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T00:45:59.593 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T00:45:59.597 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T00:45:59.618 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-03T00:45:59.648 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T00:45:59.662 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T00:45:59.674 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T00:45:59.690 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T00:45:59.722 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T00:45:59.771 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-03T00:45:59.775 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-03T00:45:59.782 INFO:teuthology.task.internal:Configuring sudo... 2024-06-03T00:45:59.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T00:45:59.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T00:45:59.789 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-03T00:45:59.848 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-03T00:45:59.856 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-03T00:45:59.857 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T00:45:59.861 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T00:45:59.863 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-03T00:45:59.897 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T00:46:00.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T00:46:00.095 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T00:46:00.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T00:46:00.229 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:46:00.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T00:46:00.314 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T00:46:00.358 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T00:46:00.415 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T00:46:00.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T00:46:00.541 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:46:00.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T00:46:00.606 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T00:46:00.649 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-03T00:46:00.708 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T00:46:00.789 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-03T00:46:00.832 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:46:00.832 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-03T00:46:00.898 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-06-03T00:46:00.903 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-06-03T00:46:00.906 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-06-03T00:46:00.942 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T00:46:00.974 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T00:46:01.001 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T00:46:01.459 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-03T00:46:01.467 INFO:teuthology.task.internal:Starting timer... 2024-06-03T00:46:01.467 INFO:teuthology.run_tasks:Running task pcp... 2024-06-03T00:46:01.477 INFO:teuthology.run_tasks:Running task selinux... 2024-06-03T00:46:01.486 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-03T00:46:01.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-06-03T00:46:01.589 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-06-03T00:46:01.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-06-03T00:46:01.661 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-06-03T00:46:01.663 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-06-03T00:46:01.735 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-06-03T00:46:01.738 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-03T00:46:01.738 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-06-03T00:46:01.792 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-06-03T00:46:01.793 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-06-03T00:46:01.857 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-06-03T00:46:01.857 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-06-03T00:46:01.882 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-06-03T00:46:01.882 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-06-03T00:46:01.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T00:46:01.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T00:46:01.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T00:46:01.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T00:46:01.981 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T00:46:02.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T00:46:02.007 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-03T00:46:02.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-06-03T00:46:02.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-06-03T00:46:02.169 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-06-03T00:46:02.203 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-03T00:46:02.213 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-03T00:46:02.216 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-03T00:46:02.342 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-03T00:46:02.355 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-03T00:46:02.356 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi107.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-03T00:52:21.385 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-06-03T00:52:21.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-06-03T00:52:21.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:52:21.492 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-06-03T00:52:21.643 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-06-03T00:52:21.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-03T00:52:21.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:52:21.718 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-06-03T00:52:21.798 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-06-03T00:52:21.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-03T00:52:21.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-03T00:52:21.874 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-06-03T00:52:21.959 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-06-03T00:52:21.960 INFO:teuthology.run_tasks:Running task clock... 2024-06-03T00:52:21.972 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-03T00:52:21.972 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T00:52:21.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T00:52:21.981 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T00:52:21.981 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T00:52:21.983 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-03T00:52:21.983 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T00:52:22.011 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T00:52:22.024 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T00:52:22.031 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T00:52:22.046 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T00:52:22.048 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-06-03T00:52:22.058 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-06-03T00:52:22.068 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-03T00:52:22.071 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-06-03T00:52:22.072 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T00:52:22.082 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-06-03T00:52:22.085 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T00:52:22.096 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T00:52:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-03T00:52:22.109 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T00:52:22.144 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-06-03T00:52:22.147 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T00:52:22.147 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-06-03T00:52:22.147 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.147 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.147 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.148 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.164 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-06-03T00:52:22.167 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T00:52:22.167 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-06-03T00:52:22.167 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.167 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.168 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.168 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.181 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-06-03T00:52:22.214 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-06-03T00:52:22.257 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-03T00:52:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-03T00:52:22.424 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-06-03T00:52:22.431 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T00:52:22.432 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-06-03T00:52:22.432 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.432 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.432 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.432 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T00:52:22.433 INFO:teuthology.run_tasks:Running task install... 2024-06-03T00:52:22.442 DEBUG:teuthology.task.install:project ceph 2024-06-03T00:52:22.442 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-03T00:52:22.442 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-03T00:52:22.442 INFO:teuthology.task.install:Using flavor: default 2024-06-03T00:52:22.449 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-03T00:52:22.449 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-03T00:52:22.450 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T00:52:22.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:52:22.453 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T00:52:22.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:52:22.455 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-03T00:52:22.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:52:22.604 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T00:52:22.605 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T00:52:22.612 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T00:52:22.612 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T00:52:22.616 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-03T00:52:22.617 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-03T00:52:22.719 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T00:52:22.719 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:52:22.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T00:52:22.724 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T00:52:22.724 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:52:22.724 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T00:52:22.750 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T00:52:22.750 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T00:52:22.784 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-03T00:52:22.784 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:52:22.784 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-03T00:52:22.803 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T00:52:22.803 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T00:52:22.809 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-03T00:52:22.810 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-03T00:52:22.816 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T00:52:22.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T00:52:22.891 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T00:52:22.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-03T00:52:22.948 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T00:52:22.954 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-06-03T00:52:22.956 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-06-03T00:52:23.011 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-06-03T00:52:23.013 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-06-03T00:52:23.040 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-03T00:52:23.114 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-06-03T00:52:23.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-06-03T00:52:23.184 INFO:teuthology.orchestra.run.smithi179.stdout:45 files removed 2024-06-03T00:52:23.211 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T00:52:23.234 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-06-03T00:52:23.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T00:52:23.432 INFO:teuthology.orchestra.run.smithi094.stdout:45 files removed 2024-06-03T00:52:23.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-03T00:52:23.881 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-06-03T00:52:23.915 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 212 kB/s | 83 kB 00:00 2024-06-03T00:52:24.240 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-06-03T00:52:24.341 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-06-03T00:52:24.357 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-06-03T00:52:24.558 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2024-06-03T00:52:24.708 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-06-03T00:52:24.824 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-06-03T00:52:25.291 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-06-03T00:52:25.317 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-06-03T00:52:25.500 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-03T00:52:25.990 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-06-03T00:52:27.001 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-06-03T00:52:27.476 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-06-03T00:52:28.150 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-06-03T00:52:31.127 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-06-03T00:52:31.693 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-06-03T00:52:32.610 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-06-03T00:52:32.962 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 17 kB 00:00 2024-06-03T00:52:34.081 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-06-03T00:52:34.177 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 17 kB 00:01 2024-06-03T00:52:35.261 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 17 kB 00:01 2024-06-03T00:52:35.512 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-06-03T00:52:36.322 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-06-03T00:52:39.281 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-06-03T00:52:40.628 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-06-03T00:52:41.582 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T00:52:41.583 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T00:52:41.593 INFO:teuthology.orchestra.run.smithi179.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T00:52:41.594 INFO:teuthology.orchestra.run.smithi179.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T00:52:41.595 INFO:teuthology.orchestra.run.smithi179.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T00:52:41.596 INFO:teuthology.orchestra.run.smithi179.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T00:52:41.598 INFO:teuthology.orchestra.run.smithi179.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T00:52:41.600 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T00:52:41.600 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T00:52:41.663 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-06-03T00:52:41.679 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-06-03T00:52:41.679 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-06-03T00:52:41.679 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-06-03T00:52:41.679 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-06-03T00:52:41.679 INFO:teuthology.orchestra.run.smithi179.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T00:52:41.679 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T00:52:41.680 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T00:52:41.681 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T00:52:41.682 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T00:52:41.683 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T00:52:41.684 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T00:52:41.685 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T00:52:41.686 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T00:52:41.687 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T00:52:41.688 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T00:52:41.689 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T00:52:41.690 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T00:52:41.691 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T00:52:41.692 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:52:41.693 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-06-03T00:52:41.694 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-06-03T00:52:41.694 INFO:teuthology.orchestra.run.smithi179.stdout:Install 148 Packages 2024-06-03T00:52:41.694 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 11 Packages 2024-06-03T00:52:41.694 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:52:41.694 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 225 M 2024-06-03T00:52:41.694 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-06-03T00:52:42.290 INFO:teuthology.orchestra.run.smithi179.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-06-03T00:52:42.458 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-06-03T00:52:42.649 INFO:teuthology.orchestra.run.smithi179.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 2.4 MB/s | 890 kB 00:00 2024-06-03T00:52:42.783 INFO:teuthology.orchestra.run.smithi179.stdout:(3/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 7.5 MB/s | 5.4 MB 00:00 2024-06-03T00:52:42.808 INFO:teuthology.orchestra.run.smithi179.stdout:(4/159): ceph-immutable-object-cache-19.0.0-294 913 kB/s | 145 kB 00:00 2024-06-03T00:52:42.921 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T00:52:42.922 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T00:52:42.933 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T00:52:42.933 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T00:52:42.934 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T00:52:42.935 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T00:52:42.937 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T00:52:42.939 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T00:52:42.940 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T00:52:43.003 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-06-03T00:52:43.023 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T00:52:43.024 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T00:52:43.025 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T00:52:43.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T00:52:43.027 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T00:52:43.028 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T00:52:43.029 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T00:52:43.030 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T00:52:43.031 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T00:52:43.032 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T00:52:43.033 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T00:52:43.034 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T00:52:43.035 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T00:52:43.036 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout:Install 148 Packages 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 11 Packages 2024-06-03T00:52:43.037 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:52:43.038 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 225 M 2024-06-03T00:52:43.038 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-06-03T00:52:43.042 INFO:teuthology.orchestra.run.smithi179.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 9.1 MB/s | 2.4 MB 00:00 2024-06-03T00:52:43.184 INFO:teuthology.orchestra.run.smithi179.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-06-03T00:52:43.510 INFO:teuthology.orchestra.run.smithi179.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 10 MB/s | 4.7 MB 00:00 2024-06-03T00:52:43.921 INFO:teuthology.orchestra.run.smithi107.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-06-03T00:52:44.035 INFO:teuthology.orchestra.run.smithi179.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 11 MB/s | 21 MB 00:01 2024-06-03T00:52:44.076 INFO:teuthology.orchestra.run.smithi179.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 607 kB/s | 25 kB 00:00 2024-06-03T00:52:44.427 INFO:teuthology.orchestra.run.smithi179.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 11 MB/s | 10 MB 00:00 2024-06-03T00:52:44.471 INFO:teuthology.orchestra.run.smithi107.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.6 MB/s | 890 kB 00:00 2024-06-03T00:52:44.552 INFO:teuthology.orchestra.run.smithi179.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.7 MB/s | 221 kB 00:00 2024-06-03T00:52:44.596 INFO:teuthology.orchestra.run.smithi107.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.1 MB/s | 145 kB 00:00 2024-06-03T00:52:44.619 INFO:teuthology.orchestra.run.smithi179.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 406 kB/s | 27 kB 00:00 2024-06-03T00:52:44.694 INFO:teuthology.orchestra.run.smithi179.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 423 kB/s | 32 kB 00:00 2024-06-03T00:52:44.870 INFO:teuthology.orchestra.run.smithi179.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.2 MB/s | 747 kB 00:00 2024-06-03T00:52:44.945 INFO:teuthology.orchestra.run.smithi179.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-03T00:52:45.020 INFO:teuthology.orchestra.run.smithi179.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.6 MB/s | 127 kB 00:00 2024-06-03T00:52:45.121 INFO:teuthology.orchestra.run.smithi179.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 4.9 MB/s | 501 kB 00:00 2024-06-03T00:52:45.133 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T00:52:45.135 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-03T00:52:45.155 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-03T00:52:45.156 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-03T00:52:45.158 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-03T00:52:45.159 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-03T00:52:45.161 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-03T00:52:45.165 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-03T00:52:45.166 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-03T00:52:45.238 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-06-03T00:52:45.263 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-06-03T00:52:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-06-03T00:52:45.263 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-03T00:52:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-03T00:52:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-03T00:52:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-03T00:52:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-03T00:52:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-03T00:52:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-03T00:52:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-03T00:52:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-03T00:52:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-03T00:52:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-03T00:52:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-03T00:52:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-03T00:52:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 11 Packages 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:52:45.277 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 225 M 2024-06-03T00:52:45.278 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-06-03T00:52:45.705 INFO:teuthology.orchestra.run.smithi107.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-06-03T00:52:45.772 INFO:teuthology.orchestra.run.smithi179.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 8.2 MB/s | 5.3 MB 00:00 2024-06-03T00:52:45.847 INFO:teuthology.orchestra.run.smithi179.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 603 kB/s | 45 kB 00:00 2024-06-03T00:52:45.900 INFO:teuthology.orchestra.run.smithi094.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-06-03T00:52:45.930 INFO:teuthology.orchestra.run.smithi179.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-03T00:52:46.014 INFO:teuthology.orchestra.run.smithi179.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-03T00:52:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-06-03T00:52:46.106 INFO:teuthology.orchestra.run.smithi179.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-03T00:52:46.163 INFO:teuthology.orchestra.run.smithi107.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-06-03T00:52:46.206 INFO:teuthology.orchestra.run.smithi179.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 300 kB 00:00 2024-06-03T00:52:46.307 INFO:teuthology.orchestra.run.smithi179.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 994 kB/s | 99 kB 00:00 2024-06-03T00:52:46.390 INFO:teuthology.orchestra.run.smithi179.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-03T00:52:46.949 INFO:teuthology.orchestra.run.smithi179.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 5.5 MB/s | 3.1 MB 00:00 2024-06-03T00:52:47.033 INFO:teuthology.orchestra.run.smithi179.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-03T00:52:47.074 INFO:teuthology.orchestra.run.smithi094.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 758 kB/s | 890 kB 00:01 2024-06-03T00:52:47.125 INFO:teuthology.orchestra.run.smithi179.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 345 kB/s | 32 kB 00:00 2024-06-03T00:52:47.209 INFO:teuthology.orchestra.run.smithi179.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-03T00:52:47.249 INFO:teuthology.orchestra.run.smithi094.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 831 kB/s | 145 kB 00:00 2024-06-03T00:52:47.851 INFO:teuthology.orchestra.run.smithi179.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 5.5 MB/s | 3.5 MB 00:00 2024-06-03T00:52:48.022 INFO:teuthology.orchestra.run.smithi107.stdout:(7/159): ceph-common-19.0.0-2946.g3d89accf.el9. 4.8 MB/s | 21 MB 00:04 2024-06-03T00:52:48.150 INFO:teuthology.orchestra.run.smithi094.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-06-03T00:52:48.185 INFO:teuthology.orchestra.run.smithi179.stdout:(31/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.4 MB/s | 17 MB 00:04 2024-06-03T00:52:48.542 INFO:teuthology.orchestra.run.smithi094.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-06-03T00:52:48.609 INFO:teuthology.orchestra.run.smithi094.stdout:(6/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-06-03T00:52:48.693 INFO:teuthology.orchestra.run.smithi179.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 489 kB/s | 249 kB 00:00 2024-06-03T00:52:48.864 INFO:teuthology.orchestra.run.smithi107.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 1.7 MB/s | 4.7 MB 00:02 2024-06-03T00:52:48.893 INFO:teuthology.orchestra.run.smithi179.stdout:(33/159): ceph-mgr-diskprediction-local-19.0.0- 7.1 MB/s | 7.4 MB 00:01 2024-06-03T00:52:48.919 INFO:teuthology.orchestra.run.smithi179.stdout:(34/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 219 kB/s | 49 kB 00:00 2024-06-03T00:52:48.930 INFO:teuthology.orchestra.run.smithi107.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 379 kB/s | 25 kB 00:00 2024-06-03T00:52:48.952 INFO:teuthology.orchestra.run.smithi179.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 281 kB/s | 16 kB 00:00 2024-06-03T00:52:49.120 INFO:teuthology.orchestra.run.smithi179.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.5 MB/s | 763 kB 00:00 2024-06-03T00:52:49.195 INFO:teuthology.orchestra.run.smithi179.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 243 kB/s | 18 kB 00:00 2024-06-03T00:52:49.278 INFO:teuthology.orchestra.run.smithi179.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 965 kB/s | 80 kB 00:00 2024-06-03T00:52:49.301 INFO:teuthology.orchestra.run.smithi094.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 6.2 MB/s | 4.7 MB 00:00 2024-06-03T00:52:49.354 INFO:teuthology.orchestra.run.smithi179.stdout:(39/159): ceph-volume-19.0.0-2946.g3d89accf.el9 606 kB/s | 264 kB 00:00 2024-06-03T00:52:49.387 INFO:teuthology.orchestra.run.smithi179.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-06-03T00:52:49.430 INFO:teuthology.orchestra.run.smithi179.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-06-03T00:52:49.488 INFO:teuthology.orchestra.run.smithi179.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-06-03T00:52:49.514 INFO:teuthology.orchestra.run.smithi179.stdout:(43/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-06-03T00:52:49.539 INFO:teuthology.orchestra.run.smithi179.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-06-03T00:52:49.564 INFO:teuthology.orchestra.run.smithi179.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-06-03T00:52:49.590 INFO:teuthology.orchestra.run.smithi179.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-03T00:52:49.635 INFO:teuthology.orchestra.run.smithi179.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.0 MB/s | 215 kB 00:00 2024-06-03T00:52:49.665 INFO:teuthology.orchestra.run.smithi179.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-06-03T00:52:49.707 INFO:teuthology.orchestra.run.smithi179.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-06-03T00:52:49.741 INFO:teuthology.orchestra.run.smithi179.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 8.7 MB/s | 943 kB 00:00 2024-06-03T00:52:49.766 INFO:teuthology.orchestra.run.smithi179.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-06-03T00:52:49.791 INFO:teuthology.orchestra.run.smithi179.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-03T00:52:49.817 INFO:teuthology.orchestra.run.smithi179.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 648 kB/s | 33 kB 00:00 2024-06-03T00:52:49.859 INFO:teuthology.orchestra.run.smithi179.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-06-03T00:52:49.918 INFO:teuthology.orchestra.run.smithi179.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-06-03T00:52:49.944 INFO:teuthology.orchestra.run.smithi179.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 132 kB/s | 11 kB 00:00 2024-06-03T00:52:49.968 INFO:teuthology.orchestra.run.smithi179.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-03T00:52:49.994 INFO:teuthology.orchestra.run.smithi179.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-06-03T00:52:50.030 INFO:teuthology.orchestra.run.smithi179.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-03T00:52:50.047 INFO:teuthology.orchestra.run.smithi107.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 5.1 MB/s | 10 MB 00:02 2024-06-03T00:52:50.053 INFO:teuthology.orchestra.run.smithi179.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-06-03T00:52:50.078 INFO:teuthology.orchestra.run.smithi179.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 505 kB/s | 25 kB 00:00 2024-06-03T00:52:50.104 INFO:teuthology.orchestra.run.smithi179.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-06-03T00:52:50.129 INFO:teuthology.orchestra.run.smithi179.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-03T00:52:50.139 INFO:teuthology.orchestra.run.smithi107.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.4 MB/s | 221 kB 00:00 2024-06-03T00:52:50.154 INFO:teuthology.orchestra.run.smithi179.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-03T00:52:50.180 INFO:teuthology.orchestra.run.smithi179.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-06-03T00:52:50.214 INFO:teuthology.orchestra.run.smithi107.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 360 kB/s | 27 kB 00:00 2024-06-03T00:52:50.265 INFO:teuthology.orchestra.run.smithi107.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 634 kB/s | 32 kB 00:00 2024-06-03T00:52:50.280 INFO:teuthology.orchestra.run.smithi179.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-06-03T00:52:50.318 INFO:teuthology.orchestra.run.smithi094.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 10 MB/s | 10 MB 00:01 2024-06-03T00:52:50.347 INFO:teuthology.orchestra.run.smithi179.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-06-03T00:52:50.382 INFO:teuthology.orchestra.run.smithi107.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 6.3 MB/s | 747 kB 00:00 2024-06-03T00:52:50.398 INFO:teuthology.orchestra.run.smithi179.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-03T00:52:50.401 INFO:teuthology.orchestra.run.smithi094.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 304 kB/s | 25 kB 00:00 2024-06-03T00:52:50.431 INFO:teuthology.orchestra.run.smithi179.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-06-03T00:52:50.432 INFO:teuthology.orchestra.run.smithi107.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 3.2 MB/s | 163 kB 00:00 2024-06-03T00:52:50.456 INFO:teuthology.orchestra.run.smithi179.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-03T00:52:50.515 INFO:teuthology.orchestra.run.smithi179.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-06-03T00:52:50.524 INFO:teuthology.orchestra.run.smithi107.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.3 MB/s | 127 kB 00:00 2024-06-03T00:52:50.591 INFO:teuthology.orchestra.run.smithi179.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-06-03T00:52:50.633 INFO:teuthology.orchestra.run.smithi179.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-06-03T00:52:50.674 INFO:teuthology.orchestra.run.smithi179.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-03T00:52:50.699 INFO:teuthology.orchestra.run.smithi107.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.8 MB/s | 501 kB 00:00 2024-06-03T00:52:50.758 INFO:teuthology.orchestra.run.smithi179.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-06-03T00:52:50.783 INFO:teuthology.orchestra.run.smithi179.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-03T00:52:50.809 INFO:teuthology.orchestra.run.smithi179.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-03T00:52:50.834 INFO:teuthology.orchestra.run.smithi179.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 508 kB/s | 26 kB 00:00 2024-06-03T00:52:50.860 INFO:teuthology.orchestra.run.smithi179.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 379 kB/s | 19 kB 00:00 2024-06-03T00:52:50.894 INFO:teuthology.orchestra.run.smithi179.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-03T00:52:50.919 INFO:teuthology.orchestra.run.smithi179.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-06-03T00:52:51.361 INFO:teuthology.orchestra.run.smithi179.stdout:(82/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 6.6 MB/s | 48 MB 00:07 2024-06-03T00:52:51.386 INFO:teuthology.orchestra.run.smithi179.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 536 kB/s | 250 kB 00:00 2024-06-03T00:52:51.429 INFO:teuthology.orchestra.run.smithi179.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-06-03T00:52:51.512 INFO:teuthology.orchestra.run.smithi179.stdout:(85/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-06-03T00:52:51.537 INFO:teuthology.orchestra.run.smithi179.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-06-03T00:52:51.563 INFO:teuthology.orchestra.run.smithi179.stdout:(87/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-06-03T00:52:51.588 INFO:teuthology.orchestra.run.smithi179.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-03T00:52:51.622 INFO:teuthology.orchestra.run.smithi179.stdout:(89/159): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-06-03T00:52:51.638 INFO:teuthology.orchestra.run.smithi179.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-03T00:52:51.655 INFO:teuthology.orchestra.run.smithi179.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-06-03T00:52:51.681 INFO:teuthology.orchestra.run.smithi179.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-06-03T00:52:51.698 INFO:teuthology.orchestra.run.smithi179.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 993 kB/s | 42 kB 00:00 2024-06-03T00:52:51.715 INFO:teuthology.orchestra.run.smithi179.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-06-03T00:52:51.740 INFO:teuthology.orchestra.run.smithi179.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-06-03T00:52:51.765 INFO:teuthology.orchestra.run.smithi179.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-06-03T00:52:51.791 INFO:teuthology.orchestra.run.smithi179.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-06-03T00:52:51.816 INFO:teuthology.orchestra.run.smithi179.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-03T00:52:51.841 INFO:teuthology.orchestra.run.smithi179.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-03T00:52:51.867 INFO:teuthology.orchestra.run.smithi179.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-03T00:52:51.892 INFO:teuthology.orchestra.run.smithi179.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-03T00:52:51.918 INFO:teuthology.orchestra.run.smithi179.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-06-03T00:52:52.010 INFO:teuthology.orchestra.run.smithi179.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-06-03T00:52:52.185 INFO:teuthology.orchestra.run.smithi179.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-06-03T00:52:52.344 INFO:teuthology.orchestra.run.smithi179.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-06-03T00:52:52.358 INFO:teuthology.orchestra.run.smithi107.stdout:(18/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.7 MB/s | 17 MB 00:06 2024-06-03T00:52:52.419 INFO:teuthology.orchestra.run.smithi179.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-06-03T00:52:52.453 INFO:teuthology.orchestra.run.smithi179.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-06-03T00:52:52.466 INFO:teuthology.orchestra.run.smithi107.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 417 kB/s | 45 kB 00:00 2024-06-03T00:52:52.478 INFO:teuthology.orchestra.run.smithi179.stdout:(108/159): grpc-data-1.46.7-10.el9.noarch.rpm 145 kB/s | 19 kB 00:00 2024-06-03T00:52:52.520 INFO:teuthology.orchestra.run.smithi179.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 387 kB/s | 26 kB 00:00 2024-06-03T00:52:52.579 INFO:teuthology.orchestra.run.smithi179.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-06-03T00:52:52.604 INFO:teuthology.orchestra.run.smithi179.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-06-03T00:52:52.666 INFO:teuthology.orchestra.run.smithi107.stdout:(20/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 2.7 MB/s | 5.3 MB 00:01 2024-06-03T00:52:52.671 INFO:teuthology.orchestra.run.smithi179.stdout:(112/159): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-06-03T00:52:52.692 INFO:teuthology.orchestra.run.smithi107.stdout:(21/159): python3-ceph-common-19.0.0-2946.g3d89 600 kB/s | 135 kB 00:00 2024-06-03T00:52:52.713 INFO:teuthology.orchestra.run.smithi179.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-06-03T00:52:52.739 INFO:teuthology.orchestra.run.smithi179.stdout:(114/159): luarocks-3.9.2-5.el9.noarch.rpm 950 kB/s | 151 kB 00:00 2024-06-03T00:52:52.750 INFO:teuthology.orchestra.run.smithi107.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.9 MB/s | 162 kB 00:00 2024-06-03T00:52:52.764 INFO:teuthology.orchestra.run.smithi179.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-06-03T00:52:52.789 INFO:teuthology.orchestra.run.smithi179.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 575 kB/s | 43 kB 00:00 2024-06-03T00:52:52.815 INFO:teuthology.orchestra.run.smithi179.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 424 kB/s | 32 kB 00:00 2024-06-03T00:52:52.840 INFO:teuthology.orchestra.run.smithi179.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-06-03T00:52:52.865 INFO:teuthology.orchestra.run.smithi179.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-06-03T00:52:52.908 INFO:teuthology.orchestra.run.smithi179.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-06-03T00:52:52.909 INFO:teuthology.orchestra.run.smithi107.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 1.8 MB/s | 300 kB 00:00 2024-06-03T00:52:52.933 INFO:teuthology.orchestra.run.smithi179.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-06-03T00:52:52.951 INFO:teuthology.orchestra.run.smithi107.stdout:(24/159): python3-rados-19.0.0-2946.g3d89accf.e 1.2 MB/s | 321 kB 00:00 2024-06-03T00:52:52.958 INFO:teuthology.orchestra.run.smithi179.stdout:(122/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-06-03T00:52:52.976 INFO:teuthology.orchestra.run.smithi107.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.4 MB/s | 99 kB 00:00 2024-06-03T00:52:53.008 INFO:teuthology.orchestra.run.smithi179.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-06-03T00:52:53.010 INFO:teuthology.orchestra.run.smithi107.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-06-03T00:52:53.034 INFO:teuthology.orchestra.run.smithi179.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2024-06-03T00:52:53.059 INFO:teuthology.orchestra.run.smithi179.stdout:(125/159): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-06-03T00:52:53.084 INFO:teuthology.orchestra.run.smithi179.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 233 kB/s | 18 kB 00:00 2024-06-03T00:52:53.110 INFO:teuthology.orchestra.run.smithi179.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 306 kB/s | 23 kB 00:00 2024-06-03T00:52:53.118 INFO:teuthology.orchestra.run.smithi107.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.5 MB/s | 171 kB 00:00 2024-06-03T00:52:53.135 INFO:teuthology.orchestra.run.smithi179.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 256 kB/s | 19 kB 00:00 2024-06-03T00:52:53.161 INFO:teuthology.orchestra.run.smithi179.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 259 kB/s | 20 kB 00:00 2024-06-03T00:52:53.185 INFO:teuthology.orchestra.run.smithi107.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 476 kB/s | 32 kB 00:00 2024-06-03T00:52:53.194 INFO:teuthology.orchestra.run.smithi179.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 790 kB/s | 46 kB 00:00 2024-06-03T00:52:53.228 INFO:teuthology.orchestra.run.smithi179.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-06-03T00:52:53.253 INFO:teuthology.orchestra.run.smithi179.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 853 kB/s | 79 kB 00:00 2024-06-03T00:52:53.279 INFO:teuthology.orchestra.run.smithi179.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-06-03T00:52:53.286 INFO:teuthology.orchestra.run.smithi107.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.4 MB/s | 142 kB 00:00 2024-06-03T00:52:53.312 INFO:teuthology.orchestra.run.smithi179.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-06-03T00:52:53.338 INFO:teuthology.orchestra.run.smithi179.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-06-03T00:52:53.363 INFO:teuthology.orchestra.run.smithi179.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-06-03T00:52:53.388 INFO:teuthology.orchestra.run.smithi179.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-06-03T00:52:53.414 INFO:teuthology.orchestra.run.smithi179.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-06-03T00:52:53.439 INFO:teuthology.orchestra.run.smithi179.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-06-03T00:52:53.464 INFO:teuthology.orchestra.run.smithi179.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 781 kB/s | 59 kB 00:00 2024-06-03T00:52:53.478 INFO:teuthology.orchestra.run.smithi107.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 6.2 MB/s | 3.1 MB 00:00 2024-06-03T00:52:53.490 INFO:teuthology.orchestra.run.smithi179.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-06-03T00:52:53.494 INFO:teuthology.orchestra.run.smithi094.stdout:(10/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 16 MB/s | 48 MB 00:03 2024-06-03T00:52:53.515 INFO:teuthology.orchestra.run.smithi179.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-06-03T00:52:53.540 INFO:teuthology.orchestra.run.smithi179.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-03T00:52:53.565 INFO:teuthology.orchestra.run.smithi179.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-06-03T00:52:53.569 INFO:teuthology.orchestra.run.smithi094.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.9 MB/s | 221 kB 00:00 2024-06-03T00:52:53.591 INFO:teuthology.orchestra.run.smithi179.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-06-03T00:52:53.616 INFO:teuthology.orchestra.run.smithi179.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-06-03T00:52:53.633 INFO:teuthology.orchestra.run.smithi179.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-06-03T00:52:53.636 INFO:teuthology.orchestra.run.smithi094.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 411 kB/s | 27 kB 00:00 2024-06-03T00:52:53.703 INFO:teuthology.orchestra.run.smithi094.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 480 kB/s | 32 kB 00:00 2024-06-03T00:52:53.719 INFO:teuthology.orchestra.run.smithi179.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-06-03T00:52:53.842 INFO:teuthology.orchestra.run.smithi179.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 19 MB/s | 2.3 MB 00:00 2024-06-03T00:52:53.861 INFO:teuthology.orchestra.run.smithi094.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 4.6 MB/s | 747 kB 00:00 2024-06-03T00:52:53.901 INFO:teuthology.orchestra.run.smithi179.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-06-03T00:52:53.935 INFO:teuthology.orchestra.run.smithi179.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-03T00:52:53.953 INFO:teuthology.orchestra.run.smithi094.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 1.8 MB/s | 163 kB 00:00 2024-06-03T00:52:53.979 INFO:teuthology.orchestra.run.smithi179.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.5 MB/s | 248 kB 00:00 2024-06-03T00:52:54.021 INFO:teuthology.orchestra.run.smithi179.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 883 kB/s | 37 kB 00:00 2024-06-03T00:52:54.037 INFO:teuthology.orchestra.run.smithi094.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.5 MB/s | 127 kB 00:00 2024-06-03T00:52:54.063 INFO:teuthology.orchestra.run.smithi179.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-03T00:52:54.121 INFO:teuthology.orchestra.run.smithi179.stdout:(155/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-06-03T00:52:54.146 INFO:teuthology.orchestra.run.smithi094.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 4.6 MB/s | 501 kB 00:00 2024-06-03T00:52:54.147 INFO:teuthology.orchestra.run.smithi179.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.0 MB/s | 260 kB 00:00 2024-06-03T00:52:54.222 INFO:teuthology.orchestra.run.smithi179.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-03T00:52:54.297 INFO:teuthology.orchestra.run.smithi179.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-06-03T00:52:54.605 INFO:teuthology.orchestra.run.smithi094.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 12 MB/s | 5.3 MB 00:00 2024-06-03T00:52:54.688 INFO:teuthology.orchestra.run.smithi094.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 548 kB/s | 45 kB 00:00 2024-06-03T00:52:54.745 INFO:teuthology.orchestra.run.smithi107.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-06-03T00:52:54.772 INFO:teuthology.orchestra.run.smithi094.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-03T00:52:54.811 INFO:teuthology.orchestra.run.smithi107.stdout:(32/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 2.3 MB/s | 3.5 MB 00:01 2024-06-03T00:52:54.853 INFO:teuthology.orchestra.run.smithi107.stdout:(33/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.2 MB/s | 249 kB 00:00 2024-06-03T00:52:54.887 INFO:teuthology.orchestra.run.smithi094.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.4 MB/s | 162 kB 00:00 2024-06-03T00:52:54.887 INFO:teuthology.orchestra.run.smithi107.stdout:(34/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 655 kB/s | 49 kB 00:00 2024-06-03T00:52:54.912 INFO:teuthology.orchestra.run.smithi107.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 281 kB/s | 16 kB 00:00 2024-06-03T00:52:54.998 INFO:teuthology.orchestra.run.smithi094.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.9 MB/s | 321 kB 00:00 2024-06-03T00:52:55.029 INFO:teuthology.orchestra.run.smithi107.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 1.8 MB/s | 264 kB 00:00 2024-06-03T00:52:55.063 INFO:teuthology.orchestra.run.smithi107.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 5.0 MB/s | 763 kB 00:00 2024-06-03T00:52:55.090 INFO:teuthology.orchestra.run.smithi094.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 3.2 MB/s | 300 kB 00:00 2024-06-03T00:52:55.096 INFO:teuthology.orchestra.run.smithi107.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 272 kB/s | 18 kB 00:00 2024-06-03T00:52:55.157 INFO:teuthology.orchestra.run.smithi094.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-03T00:52:55.172 INFO:teuthology.orchestra.run.smithi107.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 739 kB/s | 80 kB 00:00 2024-06-03T00:52:55.222 INFO:teuthology.orchestra.run.smithi107.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-06-03T00:52:55.241 INFO:teuthology.orchestra.run.smithi094.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-03T00:52:55.256 INFO:teuthology.orchestra.run.smithi107.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-06-03T00:52:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-06-03T00:52:55.323 INFO:teuthology.orchestra.run.smithi107.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-06-03T00:52:55.348 INFO:teuthology.orchestra.run.smithi107.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-06-03T00:52:55.373 INFO:teuthology.orchestra.run.smithi107.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-06-03T00:52:55.407 INFO:teuthology.orchestra.run.smithi107.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-03T00:52:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-06-03T00:52:55.466 INFO:teuthology.orchestra.run.smithi094.stdout:(26/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.4 MB/s | 17 MB 00:06 2024-06-03T00:52:55.475 INFO:teuthology.orchestra.run.smithi107.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-06-03T00:52:55.514 INFO:teuthology.orchestra.run.smithi179.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 1.8 MB/s | 3.4 MB 00:01 2024-06-03T00:52:55.517 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-06-03T00:52:55.517 INFO:teuthology.orchestra.run.smithi179.stdout:Total 16 MB/s | 225 MB 00:13 2024-06-03T00:52:55.525 INFO:teuthology.orchestra.run.smithi107.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-06-03T00:52:55.551 INFO:teuthology.orchestra.run.smithi107.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-06-03T00:52:55.567 INFO:teuthology.orchestra.run.smithi107.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 519 kB/s | 22 kB 00:00 2024-06-03T00:52:55.593 INFO:teuthology.orchestra.run.smithi107.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-06-03T00:52:55.609 INFO:teuthology.orchestra.run.smithi107.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 778 kB/s | 33 kB 00:00 2024-06-03T00:52:55.616 INFO:teuthology.orchestra.run.smithi094.stdout:(27/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 8.3 MB/s | 3.1 MB 00:00 2024-06-03T00:52:55.635 INFO:teuthology.orchestra.run.smithi107.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 733 kB/s | 18 kB 00:00 2024-06-03T00:52:55.650 INFO:teuthology.orchestra.run.smithi094.stdout:(28/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 935 kB/s | 171 kB 00:00 2024-06-03T00:52:55.684 INFO:teuthology.orchestra.run.smithi094.stdout:(29/159): ceph-grafana-dashboards-19.0.0-2946.g 480 kB/s | 32 kB 00:00 2024-06-03T00:52:55.710 INFO:teuthology.orchestra.run.smithi107.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-06-03T00:52:55.726 INFO:teuthology.orchestra.run.smithi094.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.9 MB/s | 142 kB 00:00 2024-06-03T00:52:55.735 INFO:teuthology.orchestra.run.smithi107.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-06-03T00:52:55.761 INFO:teuthology.orchestra.run.smithi107.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-03T00:52:55.786 INFO:teuthology.orchestra.run.smithi107.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-06-03T00:52:55.820 INFO:teuthology.orchestra.run.smithi107.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-03T00:52:55.845 INFO:teuthology.orchestra.run.smithi107.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-06-03T00:52:55.870 INFO:teuthology.orchestra.run.smithi107.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-06-03T00:52:55.895 INFO:teuthology.orchestra.run.smithi107.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-06-03T00:52:55.921 INFO:teuthology.orchestra.run.smithi107.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-03T00:52:55.946 INFO:teuthology.orchestra.run.smithi107.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-03T00:52:55.968 INFO:teuthology.orchestra.run.smithi094.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 12 MB/s | 3.5 MB 00:00 2024-06-03T00:52:55.971 INFO:teuthology.orchestra.run.smithi107.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-06-03T00:52:56.044 INFO:teuthology.orchestra.run.smithi094.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 3.3 MB/s | 249 kB 00:00 2024-06-03T00:52:56.080 INFO:teuthology.orchestra.run.smithi107.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-06-03T00:52:56.102 INFO:teuthology.orchestra.run.smithi094.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 858 kB/s | 49 kB 00:00 2024-06-03T00:52:56.147 INFO:teuthology.orchestra.run.smithi107.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-06-03T00:52:56.152 INFO:teuthology.orchestra.run.smithi094.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 333 kB/s | 16 kB 00:00 2024-06-03T00:52:56.189 INFO:teuthology.orchestra.run.smithi107.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-03T00:52:56.223 INFO:teuthology.orchestra.run.smithi107.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-06-03T00:52:56.228 INFO:teuthology.orchestra.run.smithi094.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.5 MB/s | 264 kB 00:00 2024-06-03T00:52:56.248 INFO:teuthology.orchestra.run.smithi107.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-03T00:52:56.298 INFO:teuthology.orchestra.run.smithi107.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-06-03T00:52:56.336 INFO:teuthology.orchestra.run.smithi094.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 6.9 MB/s | 763 kB 00:00 2024-06-03T00:52:56.382 INFO:teuthology.orchestra.run.smithi107.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-06-03T00:52:56.395 INFO:teuthology.orchestra.run.smithi094.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 314 kB/s | 18 kB 00:00 2024-06-03T00:52:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-06-03T00:52:56.458 INFO:teuthology.orchestra.run.smithi107.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-06-03T00:52:56.504 INFO:teuthology.orchestra.run.smithi094.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 748 kB/s | 80 kB 00:00 2024-06-03T00:52:56.541 INFO:teuthology.orchestra.run.smithi107.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-06-03T00:52:56.567 INFO:teuthology.orchestra.run.smithi107.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-03T00:52:56.592 INFO:teuthology.orchestra.run.smithi107.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-06-03T00:52:56.617 INFO:teuthology.orchestra.run.smithi107.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-03T00:52:56.629 INFO:teuthology.orchestra.run.smithi094.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-06-03T00:52:56.642 INFO:teuthology.orchestra.run.smithi107.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-06-03T00:52:56.668 INFO:teuthology.orchestra.run.smithi107.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-03T00:52:56.680 INFO:teuthology.orchestra.run.smithi094.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-06-03T00:52:56.693 INFO:teuthology.orchestra.run.smithi107.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-06-03T00:52:56.718 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-06-03T00:52:56.730 INFO:teuthology.orchestra.run.smithi094.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-06-03T00:52:56.735 INFO:teuthology.orchestra.run.smithi107.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-03T00:52:56.814 INFO:teuthology.orchestra.run.smithi094.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-06-03T00:52:56.818 INFO:teuthology.orchestra.run.smithi107.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-06-03T00:52:56.843 INFO:teuthology.orchestra.run.smithi107.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-06-03T00:52:56.864 INFO:teuthology.orchestra.run.smithi094.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-06-03T00:52:56.869 INFO:teuthology.orchestra.run.smithi107.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-03T00:52:56.876 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-06-03T00:52:56.876 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-06-03T00:52:56.894 INFO:teuthology.orchestra.run.smithi107.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-06-03T00:52:56.914 INFO:teuthology.orchestra.run.smithi094.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-06-03T00:52:56.919 INFO:teuthology.orchestra.run.smithi107.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-06-03T00:52:56.945 INFO:teuthology.orchestra.run.smithi107.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-03T00:52:56.978 INFO:teuthology.orchestra.run.smithi107.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-06-03T00:52:56.982 INFO:teuthology.orchestra.run.smithi094.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-06-03T00:52:57.021 INFO:teuthology.orchestra.run.smithi107.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-06-03T00:52:57.046 INFO:teuthology.orchestra.run.smithi107.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 803 kB/s | 54 kB 00:00 2024-06-03T00:52:57.048 INFO:teuthology.orchestra.run.smithi094.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-06-03T00:52:57.088 INFO:teuthology.orchestra.run.smithi107.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-03T00:52:57.116 INFO:teuthology.orchestra.run.smithi094.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-06-03T00:52:57.130 INFO:teuthology.orchestra.run.smithi107.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-03T00:52:57.182 INFO:teuthology.orchestra.run.smithi094.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-06-03T00:52:57.189 INFO:teuthology.orchestra.run.smithi107.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-06-03T00:52:57.230 INFO:teuthology.orchestra.run.smithi107.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 868 kB/s | 36 kB 00:00 2024-06-03T00:52:57.342 INFO:teuthology.orchestra.run.smithi094.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 675 kB/s | 107 kB 00:00 2024-06-03T00:52:57.392 INFO:teuthology.orchestra.run.smithi094.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 441 kB/s | 22 kB 00:00 2024-06-03T00:52:57.406 INFO:teuthology.orchestra.run.smithi107.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-06-03T00:52:57.431 INFO:teuthology.orchestra.run.smithi107.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 237 kB/s | 47 kB 00:00 2024-06-03T00:52:57.442 INFO:teuthology.orchestra.run.smithi094.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.3 MB/s | 313 kB 00:00 2024-06-03T00:52:57.456 INFO:teuthology.orchestra.run.smithi107.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-06-03T00:52:57.481 INFO:teuthology.orchestra.run.smithi107.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-03T00:52:57.493 INFO:teuthology.orchestra.run.smithi094.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-06-03T00:52:57.506 INFO:teuthology.orchestra.run.smithi107.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-03T00:52:57.532 INFO:teuthology.orchestra.run.smithi107.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-03T00:52:57.558 INFO:teuthology.orchestra.run.smithi107.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 887 kB/s | 45 kB 00:00 2024-06-03T00:52:57.653 INFO:teuthology.orchestra.run.smithi094.stdout:(53/159): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-06-03T00:52:57.675 INFO:teuthology.orchestra.run.smithi107.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-06-03T00:52:57.760 INFO:teuthology.orchestra.run.smithi094.stdout:(54/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-06-03T00:52:57.777 INFO:teuthology.orchestra.run.smithi094.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 18 kB 00:00 2024-06-03T00:52:57.794 INFO:teuthology.orchestra.run.smithi094.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 340 kB/s | 11 kB 00:00 2024-06-03T00:52:57.825 INFO:teuthology.orchestra.run.smithi107.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-06-03T00:52:57.845 INFO:teuthology.orchestra.run.smithi094.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 917 kB/s | 45 kB 00:00 2024-06-03T00:52:57.895 INFO:teuthology.orchestra.run.smithi094.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-06-03T00:52:57.937 INFO:teuthology.orchestra.run.smithi094.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-06-03T00:52:58.021 INFO:teuthology.orchestra.run.smithi094.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-06-03T00:52:58.046 INFO:teuthology.orchestra.run.smithi094.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-06-03T00:52:58.080 INFO:teuthology.orchestra.run.smithi094.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 819 kB/s | 47 kB 00:00 2024-06-03T00:52:58.113 INFO:teuthology.orchestra.run.smithi094.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-06-03T00:52:58.147 INFO:teuthology.orchestra.run.smithi094.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-06-03T00:52:58.172 INFO:teuthology.orchestra.run.smithi094.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 607 kB/s | 35 kB 00:00 2024-06-03T00:52:58.275 INFO:teuthology.orchestra.run.smithi107.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 913 kB/s | 548 kB 00:00 2024-06-03T00:52:58.342 INFO:teuthology.orchestra.run.smithi107.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-06-03T00:52:58.364 INFO:teuthology.orchestra.run.smithi094.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-06-03T00:52:58.384 INFO:teuthology.orchestra.run.smithi107.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-06-03T00:52:58.465 INFO:teuthology.orchestra.run.smithi094.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-06-03T00:52:58.526 INFO:teuthology.orchestra.run.smithi107.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-06-03T00:52:58.574 INFO:teuthology.orchestra.run.smithi094.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-06-03T00:52:58.680 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-06-03T00:52:58.680 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-06-03T00:52:58.686 INFO:teuthology.orchestra.run.smithi094.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-06-03T00:52:58.725 INFO:teuthology.orchestra.run.smithi094.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 892 kB/s | 133 kB 00:00 2024-06-03T00:52:58.808 INFO:teuthology.orchestra.run.smithi094.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-06-03T00:52:58.859 INFO:teuthology.orchestra.run.smithi094.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-06-03T00:52:58.976 INFO:teuthology.orchestra.run.smithi094.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-06-03T00:52:59.026 INFO:teuthology.orchestra.run.smithi107.stdout:(109/159): ceph-test-19.0.0-2946.g3d89accf.el9. 4.8 MB/s | 48 MB 00:10 2024-06-03T00:52:59.051 INFO:teuthology.orchestra.run.smithi094.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 824 kB/s | 61 kB 00:00 2024-06-03T00:52:59.102 INFO:teuthology.orchestra.run.smithi094.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-06-03T00:52:59.143 INFO:teuthology.orchestra.run.smithi107.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 79 kB/s | 49 kB 00:00 2024-06-03T00:52:59.152 INFO:teuthology.orchestra.run.smithi094.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-06-03T00:52:59.202 INFO:teuthology.orchestra.run.smithi094.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 519 kB/s | 26 kB 00:00 2024-06-03T00:52:59.293 INFO:teuthology.orchestra.run.smithi107.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-06-03T00:52:59.311 INFO:teuthology.orchestra.run.smithi094.stdout:(78/159): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-06-03T00:52:59.318 INFO:teuthology.orchestra.run.smithi107.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 865 kB/s | 151 kB 00:00 2024-06-03T00:52:59.336 INFO:teuthology.orchestra.run.smithi094.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-06-03T00:52:59.395 INFO:teuthology.orchestra.run.smithi094.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-06-03T00:52:59.420 INFO:teuthology.orchestra.run.smithi094.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 419 kB/s | 35 kB 00:00 2024-06-03T00:52:59.479 INFO:teuthology.orchestra.run.smithi094.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-06-03T00:52:59.519 INFO:teuthology.orchestra.run.smithi107.stdout:(113/159): libunwind-1.6.2-1.el9.x86_64.rpm 137 kB/s | 67 kB 00:00 2024-06-03T00:52:59.553 INFO:teuthology.orchestra.run.smithi107.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-06-03T00:52:59.586 INFO:teuthology.orchestra.run.smithi107.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-06-03T00:52:59.645 INFO:teuthology.orchestra.run.smithi107.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-06-03T00:52:59.670 INFO:teuthology.orchestra.run.smithi107.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-06-03T00:52:59.712 INFO:teuthology.orchestra.run.smithi107.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 119 kB/s | 15 kB 00:00 2024-06-03T00:52:59.771 INFO:teuthology.orchestra.run.smithi094.stdout:(83/159): ceph-common-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 21 MB 00:14 2024-06-03T00:52:59.787 INFO:teuthology.orchestra.run.smithi107.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-06-03T00:52:59.805 INFO:teuthology.orchestra.run.smithi094.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 684 kB/s | 222 kB 00:00 2024-06-03T00:52:59.855 INFO:teuthology.orchestra.run.smithi094.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 860 kB/s | 42 kB 00:00 2024-06-03T00:52:59.862 INFO:teuthology.orchestra.run.smithi107.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-06-03T00:52:59.896 INFO:teuthology.orchestra.run.smithi107.stdout:(121/159): python3-cheroot-8.6.0-4.el9.noarch.r 685 kB/s | 172 kB 00:00 2024-06-03T00:52:59.905 INFO:teuthology.orchestra.run.smithi094.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-06-03T00:52:59.921 INFO:teuthology.orchestra.run.smithi107.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-06-03T00:52:59.931 INFO:teuthology.orchestra.run.smithi094.stdout:(87/159): python3-packaging-20.9-5.el9.noarch.r 487 kB/s | 77 kB 00:00 2024-06-03T00:52:59.964 INFO:teuthology.orchestra.run.smithi094.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-06-03T00:53:00.030 INFO:teuthology.orchestra.run.smithi107.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-06-03T00:53:00.057 INFO:teuthology.orchestra.run.smithi107.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 910 kB/s | 144 kB 00:00 2024-06-03T00:53:00.096 INFO:teuthology.orchestra.run.smithi094.stdout:(89/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.3 MB/s | 5.1 MB 00:00 2024-06-03T00:53:00.124 INFO:teuthology.orchestra.run.smithi094.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 323 kB/s | 51 kB 00:00 2024-06-03T00:53:00.139 INFO:teuthology.orchestra.run.smithi107.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-06-03T00:53:00.149 INFO:teuthology.orchestra.run.smithi094.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-03T00:53:00.164 INFO:teuthology.orchestra.run.smithi107.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-06-03T00:53:00.182 INFO:teuthology.orchestra.run.smithi094.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-06-03T00:53:00.189 INFO:teuthology.orchestra.run.smithi107.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-06-03T00:53:00.208 INFO:teuthology.orchestra.run.smithi094.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 726 kB/s | 42 kB 00:00 2024-06-03T00:53:00.231 INFO:teuthology.orchestra.run.smithi107.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-06-03T00:53:00.233 INFO:teuthology.orchestra.run.smithi094.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 888 kB/s | 44 kB 00:00 2024-06-03T00:53:00.257 INFO:teuthology.orchestra.run.smithi107.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-06-03T00:53:00.267 INFO:teuthology.orchestra.run.smithi094.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-06-03T00:53:00.292 INFO:teuthology.orchestra.run.smithi094.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 620 kB/s | 36 kB 00:00 2024-06-03T00:53:00.318 INFO:teuthology.orchestra.run.smithi094.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-06-03T00:53:00.341 INFO:teuthology.orchestra.run.smithi107.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 425 kB/s | 46 kB 00:00 2024-06-03T00:53:00.351 INFO:teuthology.orchestra.run.smithi094.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-06-03T00:53:00.365 INFO:teuthology.orchestra.run.smithi107.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-06-03T00:53:00.377 INFO:teuthology.orchestra.run.smithi094.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 999 kB/s | 58 kB 00:00 2024-06-03T00:53:00.402 INFO:teuthology.orchestra.run.smithi094.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-03T00:53:00.448 INFO:teuthology.orchestra.run.smithi094.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 904 kB/s | 64 kB 00:00 2024-06-03T00:53:00.466 INFO:teuthology.orchestra.run.smithi107.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-06-03T00:53:00.478 INFO:teuthology.orchestra.run.smithi094.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-06-03T00:53:00.491 INFO:teuthology.orchestra.run.smithi107.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-06-03T00:53:00.525 INFO:teuthology.orchestra.run.smithi107.stdout:(134/159): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-06-03T00:53:00.545 INFO:teuthology.orchestra.run.smithi094.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 234 kB/s | 22 kB 00:00 2024-06-03T00:53:00.592 INFO:teuthology.orchestra.run.smithi107.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-06-03T00:53:00.609 INFO:teuthology.orchestra.run.smithi107.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 767 kB/s | 90 kB 00:00 2024-06-03T00:53:00.620 INFO:teuthology.orchestra.run.smithi094.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-06-03T00:53:00.667 INFO:teuthology.orchestra.run.smithi107.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 198 kB/s | 28 kB 00:00 2024-06-03T00:53:00.701 INFO:teuthology.orchestra.run.smithi107.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-06-03T00:53:00.726 INFO:teuthology.orchestra.run.smithi107.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-06-03T00:53:00.768 INFO:teuthology.orchestra.run.smithi107.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-06-03T00:53:00.802 INFO:teuthology.orchestra.run.smithi107.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-06-03T00:53:00.844 INFO:teuthology.orchestra.run.smithi107.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2024-06-03T00:53:00.885 INFO:teuthology.orchestra.run.smithi107.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-06-03T00:53:00.911 INFO:teuthology.orchestra.run.smithi107.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-06-03T00:53:00.971 INFO:teuthology.orchestra.run.smithi094.stdout:(105/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-06-03T00:53:00.978 INFO:teuthology.orchestra.run.smithi107.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-06-03T00:53:01.003 INFO:teuthology.orchestra.run.smithi107.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-06-03T00:53:01.053 INFO:teuthology.orchestra.run.smithi107.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-06-03T00:53:01.187 INFO:teuthology.orchestra.run.smithi107.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-06-03T00:53:01.271 INFO:teuthology.orchestra.run.smithi094.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 66 kB/s | 19 kB 00:00 2024-06-03T00:53:01.321 INFO:teuthology.orchestra.run.smithi094.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 482 kB/s | 309 kB 00:00 2024-06-03T00:53:01.337 INFO:teuthology.orchestra.run.smithi107.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-06-03T00:53:01.404 INFO:teuthology.orchestra.run.smithi107.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-03T00:53:01.405 INFO:teuthology.orchestra.run.smithi094.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 638 kB/s | 548 kB 00:00 2024-06-03T00:53:01.430 INFO:teuthology.orchestra.run.smithi094.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-06-03T00:53:01.463 INFO:teuthology.orchestra.run.smithi107.stdout:(151/159): librados2-19.0.0-2946.g3d89accf.el9. 7.4 MB/s | 3.4 MB 00:00 2024-06-03T00:53:01.480 INFO:teuthology.orchestra.run.smithi107.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 549 kB/s | 41 kB 00:00 2024-06-03T00:53:01.505 INFO:teuthology.orchestra.run.smithi107.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.4 MB/s | 37 kB 00:00 2024-06-03T00:53:01.514 INFO:teuthology.orchestra.run.smithi094.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 450 kB/s | 49 kB 00:00 2024-06-03T00:53:01.538 INFO:teuthology.orchestra.run.smithi107.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 6.1 MB/s | 207 kB 00:00 2024-06-03T00:53:01.539 INFO:teuthology.orchestra.run.smithi094.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 624 kB/s | 67 kB 00:00 2024-06-03T00:53:01.564 INFO:teuthology.orchestra.run.smithi107.stdout:(155/159): NetworkManager-tui-1.47.91-1.el9.x86 2.4 MB/s | 248 kB 00:00 2024-06-03T00:53:01.589 INFO:teuthology.orchestra.run.smithi107.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.0 MB/s | 260 kB 00:00 2024-06-03T00:53:01.656 INFO:teuthology.orchestra.run.smithi107.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-06-03T00:53:01.656 INFO:teuthology.orchestra.run.smithi094.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-06-03T00:53:01.698 INFO:teuthology.orchestra.run.smithi107.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-06-03T00:53:01.732 INFO:teuthology.orchestra.run.smithi094.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-06-03T00:53:01.874 INFO:teuthology.orchestra.run.smithi094.stdout:(114/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-06-03T00:53:01.899 INFO:teuthology.orchestra.run.smithi094.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 261 kB/s | 43 kB 00:00 2024-06-03T00:53:01.941 INFO:teuthology.orchestra.run.smithi094.stdout:(116/159): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-06-03T00:53:01.966 INFO:teuthology.orchestra.run.smithi094.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 352 kB/s | 32 kB 00:00 2024-06-03T00:53:01.992 INFO:teuthology.orchestra.run.smithi094.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-06-03T00:53:02.067 INFO:teuthology.orchestra.run.smithi094.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-06-03T00:53:02.092 INFO:teuthology.orchestra.run.smithi094.stdout:(120/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-06-03T00:53:02.140 INFO:teuthology.orchestra.run.smithi107.stdout:(159/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-06-03T00:53:02.143 INFO:teuthology.orchestra.run.smithi094.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-06-03T00:53:02.143 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-06-03T00:53:02.144 INFO:teuthology.orchestra.run.smithi107.stdout:Total 12 MB/s | 225 MB 00:19 2024-06-03T00:53:02.194 INFO:teuthology.orchestra.run.smithi094.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-06-03T00:53:02.235 INFO:teuthology.orchestra.run.smithi094.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-06-03T00:53:02.269 INFO:teuthology.orchestra.run.smithi094.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 145 kB/s | 11 kB 00:00 2024-06-03T00:53:02.311 INFO:teuthology.orchestra.run.smithi094.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-06-03T00:53:02.345 INFO:teuthology.orchestra.run.smithi094.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2024-06-03T00:53:02.387 INFO:teuthology.orchestra.run.smithi094.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-06-03T00:53:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-06-03T00:53:02.504 INFO:teuthology.orchestra.run.smithi094.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-06-03T00:53:02.547 INFO:teuthology.orchestra.run.smithi094.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-06-03T00:53:02.580 INFO:teuthology.orchestra.run.smithi094.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-06-03T00:53:02.622 INFO:teuthology.orchestra.run.smithi094.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 778 kB/s | 58 kB 00:00 2024-06-03T00:53:02.664 INFO:teuthology.orchestra.run.smithi094.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-06-03T00:53:02.739 INFO:teuthology.orchestra.run.smithi094.stdout:(134/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-06-03T00:53:02.765 INFO:teuthology.orchestra.run.smithi094.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-06-03T00:53:02.790 INFO:teuthology.orchestra.run.smithi094.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 719 kB/s | 90 kB 00:00 2024-06-03T00:53:02.815 INFO:teuthology.orchestra.run.smithi094.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 377 kB/s | 28 kB 00:00 2024-06-03T00:53:02.832 INFO:teuthology.orchestra.run.smithi094.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 461 kB/s | 31 kB 00:00 2024-06-03T00:53:02.899 INFO:teuthology.orchestra.run.smithi094.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-06-03T00:53:02.925 INFO:teuthology.orchestra.run.smithi094.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-06-03T00:53:02.950 INFO:teuthology.orchestra.run.smithi094.stdout:(141/159): python3-rsa-4.9-2.el9.noarch.rpm 442 kB/s | 59 kB 00:00 2024-06-03T00:53:02.975 INFO:teuthology.orchestra.run.smithi094.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-06-03T00:53:03.026 INFO:teuthology.orchestra.run.smithi094.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-06-03T00:53:03.061 INFO:teuthology.orchestra.run.smithi094.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-06-03T00:53:03.095 INFO:teuthology.orchestra.run.smithi094.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-06-03T00:53:03.120 INFO:teuthology.orchestra.run.smithi094.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 213 kB/s | 20 kB 00:00 2024-06-03T00:53:03.277 INFO:teuthology.orchestra.run.smithi094.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 894 kB/s | 192 kB 00:00 2024-06-03T00:53:03.344 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-06-03T00:53:03.346 INFO:teuthology.orchestra.run.smithi094.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-06-03T00:53:03.501 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-06-03T00:53:03.502 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-06-03T00:53:03.580 INFO:teuthology.orchestra.run.smithi094.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 9.9 MB/s | 2.3 MB 00:00 2024-06-03T00:53:03.900 INFO:teuthology.orchestra.run.smithi094.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 5.7 MB/s | 1.8 MB 00:00 2024-06-03T00:53:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:(151/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-06-03T00:53:04.039 INFO:teuthology.orchestra.run.smithi094.stdout:(152/159): NetworkManager-team-1.47.91-1.el9.x8 299 kB/s | 41 kB 00:00 2024-06-03T00:53:04.131 INFO:teuthology.orchestra.run.smithi094.stdout:(153/159): librados2-19.0.0-2946.g3d89accf.el9. 3.4 MB/s | 3.4 MB 00:01 2024-06-03T00:53:04.156 INFO:teuthology.orchestra.run.smithi094.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 316 kB/s | 37 kB 00:00 2024-06-03T00:53:04.190 INFO:teuthology.orchestra.run.smithi094.stdout:(155/159): NetworkManager-tui-1.47.91-1.el9.x86 1.4 MB/s | 248 kB 00:00 2024-06-03T00:53:04.223 INFO:teuthology.orchestra.run.smithi094.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-06-03T00:53:04.257 INFO:teuthology.orchestra.run.smithi094.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.6 MB/s | 207 kB 00:00 2024-06-03T00:53:04.341 INFO:teuthology.orchestra.run.smithi094.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-06-03T00:53:04.407 INFO:teuthology.orchestra.run.smithi094.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-06-03T00:53:04.420 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-06-03T00:53:04.420 INFO:teuthology.orchestra.run.smithi094.stdout:Total 12 MB/s | 225 MB 00:19 2024-06-03T00:53:05.025 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-06-03T00:53:05.102 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T00:53:05.139 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T00:53:05.193 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T00:53:05.227 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T00:53:05.253 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T00:53:05.287 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T00:53:05.302 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-06-03T00:53:05.302 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-06-03T00:53:05.621 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T00:53:05.737 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:05.859 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:05.897 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:05.988 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:06.017 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T00:53:06.050 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T00:53:06.242 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T00:53:06.290 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T00:53:06.318 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:06.422 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:06.453 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T00:53:06.552 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T00:53:06.721 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T00:53:06.739 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T00:53:07.004 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T00:53:07.134 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T00:53:07.284 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T00:53:07.328 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T00:53:07.422 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:07.513 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:07.593 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-06-03T00:53:07.706 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:07.813 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T00:53:07.842 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-06-03T00:53:07.842 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-06-03T00:53:07.907 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T00:53:07.943 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T00:53:07.979 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T00:53:08.044 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T00:53:08.078 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T00:53:08.121 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T00:53:08.154 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T00:53:08.190 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T00:53:08.260 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T00:53:08.313 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T00:53:08.343 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T00:53:08.378 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T00:53:08.454 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T00:53:08.489 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T00:53:08.532 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T00:53:08.561 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T00:53:08.631 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T00:53:08.676 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T00:53:08.749 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T00:53:08.885 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T00:53:08.937 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T00:53:09.455 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T00:53:09.540 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T00:53:09.570 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T00:53:09.606 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T00:53:09.655 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T00:53:09.693 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T00:53:09.734 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T00:53:09.882 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T00:53:09.930 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T00:53:09.976 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T00:53:10.041 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T00:53:10.072 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T00:53:10.109 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T00:53:10.155 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T00:53:10.191 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T00:53:10.234 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T00:53:10.276 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T00:53:10.411 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T00:53:10.414 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-06-03T00:53:10.415 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-06-03T00:53:10.462 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T00:53:10.503 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T00:53:10.597 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T00:53:10.637 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T00:53:10.676 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T00:53:10.702 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T00:53:10.727 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T00:53:10.800 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T00:53:10.929 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T00:53:10.968 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T00:53:10.987 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T00:53:11.021 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T00:53:11.061 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T00:53:11.092 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T00:53:11.202 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T00:53:11.449 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T00:53:11.483 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T00:53:11.525 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T00:53:11.558 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T00:53:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-06-03T00:53:11.650 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T00:53:11.715 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T00:53:11.744 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T00:53:11.745 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T00:53:11.782 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T00:53:11.793 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T00:53:11.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T00:53:11.877 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T00:53:11.901 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T00:53:11.903 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T00:53:11.946 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T00:53:11.954 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T00:53:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T00:53:12.379 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:12.501 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:12.522 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T00:53:12.677 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T00:53:12.684 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T00:53:12.718 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T00:53:12.752 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T00:53:12.849 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T00:53:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T00:53:12.899 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T00:53:12.924 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T00:53:12.943 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T00:53:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:13.030 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T00:53:13.047 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:13.070 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T00:53:13.088 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T00:53:13.132 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T00:53:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T00:53:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T00:53:13.266 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T00:53:13.302 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T00:53:13.321 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T00:53:13.380 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T00:53:13.447 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T00:53:13.602 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T00:53:13.646 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T00:53:13.747 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:13.830 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:14.019 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T00:53:14.040 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:14.133 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T00:53:14.153 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T00:53:14.247 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T00:53:14.278 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T00:53:14.321 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T00:53:14.385 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T00:53:14.420 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T00:53:14.464 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T00:53:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T00:53:14.524 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T00:53:14.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T00:53:14.646 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T00:53:14.677 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T00:53:14.712 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T00:53:14.788 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T00:53:14.823 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T00:53:14.866 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T00:53:14.895 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T00:53:14.965 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T00:53:15.010 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T00:53:15.084 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T00:53:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T00:53:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T00:53:15.351 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T00:53:15.431 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T00:53:15.459 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T00:53:15.492 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T00:53:15.789 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T00:53:15.804 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T00:53:15.847 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T00:53:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T00:53:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T00:53:15.921 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T00:53:15.931 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T00:53:15.952 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T00:53:15.972 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T00:53:15.988 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T00:53:16.001 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T00:53:16.051 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T00:53:16.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T00:53:16.255 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T00:53:16.311 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T00:53:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T00:53:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T00:53:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T00:53:16.462 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T00:53:16.483 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T00:53:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T00:53:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T00:53:16.555 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T00:53:16.576 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T00:53:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T00:53:16.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T00:53:16.771 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T00:53:16.822 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T00:53:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T00:53:17.049 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T00:53:17.080 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T00:53:17.111 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T00:53:17.136 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T00:53:17.162 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T00:53:17.236 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T00:53:17.363 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T00:53:17.393 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T00:53:17.421 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T00:53:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-06-03T00:53:17.455 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T00:53:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T00:53:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T00:53:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T00:53:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-03T00:53:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T00:53:17.632 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T00:53:17.674 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T00:53:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-03T00:53:17.708 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T00:53:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-03T00:53:17.794 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T00:53:17.846 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T00:53:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-03T00:53:17.878 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T00:53:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-03T00:53:17.925 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T00:53:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-03T00:53:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T00:53:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T00:53:18.452 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-03T00:53:18.593 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:18.776 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:18.788 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T00:53:18.806 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:18.821 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:18.834 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T00:53:18.856 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:18.868 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T00:53:18.942 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:18.965 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T00:53:18.980 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T00:53:19.023 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T00:53:19.049 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-03T00:53:19.067 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T00:53:19.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T00:53:19.196 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T00:53:19.351 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-03T00:53:19.372 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T00:53:19.397 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:19.402 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T00:53:19.428 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-03T00:53:19.434 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T00:53:19.453 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:19.480 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T00:53:19.534 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T00:53:19.535 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T00:53:19.535 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T00:53:19.535 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:19.561 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T00:53:19.622 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:19.654 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T00:53:19.749 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-03T00:53:19.780 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T00:53:19.838 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-03T00:53:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T00:53:20.161 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T00:53:20.240 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T00:53:20.482 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-03T00:53:20.732 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T00:53:20.816 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-03T00:53:20.969 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:21.462 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T00:53:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-03T00:53:21.538 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T00:53:21.566 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T00:53:21.599 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T00:53:21.681 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-03T00:53:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-03T00:53:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-03T00:53:21.894 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-03T00:53:21.910 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T00:53:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T00:53:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-03T00:53:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-03T00:53:22.044 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T00:53:22.074 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T00:53:22.101 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-03T00:53:22.111 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T00:53:22.146 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-03T00:53:22.194 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-03T00:53:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-03T00:53:22.376 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-03T00:53:22.413 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-03T00:53:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-03T00:53:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T00:53:22.607 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-03T00:53:22.614 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T00:53:22.650 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-03T00:53:22.669 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T00:53:22.699 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T00:53:22.705 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-03T00:53:22.740 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-03T00:53:22.859 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-03T00:53:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-03T00:53:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-03T00:53:23.227 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-03T00:53:23.296 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-03T00:53:23.992 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-03T00:53:24.102 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-03T00:53:24.154 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-03T00:53:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-03T00:53:24.285 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-03T00:53:24.349 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-03T00:53:24.421 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-03T00:53:24.637 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-03T00:53:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-03T00:53:24.803 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-03T00:53:24.894 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-03T00:53:24.960 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-03T00:53:24.977 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:25.005 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-03T00:53:25.012 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:25.064 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-03T00:53:25.103 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-03T00:53:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-03T00:53:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-03T00:53:25.426 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-03T00:53:25.481 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-03T00:53:25.536 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-03T00:53:25.551 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:25.607 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T00:53:25.664 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-03T00:53:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T00:53:25.702 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T00:53:25.703 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T00:53:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:25.722 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-03T00:53:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T00:53:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-03T00:53:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-03T00:53:25.837 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-03T00:53:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-03T00:53:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-03T00:53:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-03T00:53:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-03T00:53:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-03T00:53:26.473 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-03T00:53:26.531 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-03T00:53:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-03T00:53:26.626 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-03T00:53:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-03T00:53:26.761 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-03T00:53:26.809 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-03T00:53:26.959 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-03T00:53:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-03T00:53:27.080 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-03T00:53:27.142 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-03T00:53:27.299 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-03T00:53:27.368 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-03T00:53:28.335 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-03T00:53:28.384 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-03T00:53:28.442 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-03T00:53:28.567 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-03T00:53:28.634 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-03T00:53:28.690 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-03T00:53:28.797 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-03T00:53:28.845 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-03T00:53:29.078 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-03T00:53:29.125 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-03T00:53:29.247 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-03T00:53:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-03T00:53:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-03T00:53:31.284 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T00:53:31.284 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-06-03T00:53:31.284 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-06-03T00:53:31.285 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-06-03T00:53:31.285 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-06-03T00:53:31.285 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-06-03T00:53:31.285 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-06-03T00:53:31.285 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-06-03T00:53:31.285 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:31.452 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T00:53:31.470 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T00:53:31.470 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:31.470 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T00:53:31.470 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-03T00:53:31.952 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-03T00:53:31.982 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-03T00:53:32.015 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-03T00:53:32.365 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-03T00:53:32.410 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T00:53:32.531 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-03T00:53:32.591 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-03T00:53:32.648 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-03T00:53:32.955 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T00:53:32.981 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T00:53:32.981 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:32.981 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T00:53:32.981 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T00:53:32.982 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T00:53:32.982 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:33.188 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-03T00:53:33.206 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T00:53:33.221 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T00:53:33.228 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T00:53:33.228 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:33.228 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T00:53:33.228 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T00:53:33.228 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T00:53:33.228 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:33.303 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-03T00:53:33.331 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-03T00:53:33.673 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T00:53:33.695 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T00:53:33.696 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:33.696 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T00:53:33.696 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T00:53:33.696 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T00:53:33.696 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:33.794 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T00:53:33.853 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T00:53:33.907 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T00:53:34.092 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T00:53:34.140 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T00:53:34.218 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T00:53:34.307 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T00:53:34.446 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T00:53:34.499 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T00:53:34.530 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T00:53:34.558 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T00:53:34.952 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T00:53:34.981 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T00:53:35.013 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T00:53:35.091 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T00:53:35.121 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T00:53:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:36.156 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T00:53:36.190 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T00:53:36.711 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-03T00:53:36.755 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T00:53:36.917 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-03T00:53:36.918 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-03T00:53:36.918 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-03T00:53:36.918 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:36.956 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T00:53:37.031 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T00:53:37.031 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-06-03T00:53:37.031 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-06-03T00:53:37.032 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-06-03T00:53:37.032 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-06-03T00:53:37.032 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-06-03T00:53:37.032 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-06-03T00:53:37.032 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-06-03T00:53:37.032 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:37.153 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T00:53:37.182 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T00:53:37.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T00:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T00:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T00:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:37.283 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T00:53:37.437 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T00:53:37.476 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T00:53:37.498 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T00:53:37.499 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:37.499 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T00:53:37.499 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T00:53:37.499 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T00:53:37.499 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:37.543 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T00:53:37.554 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T00:53:37.653 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T00:53:37.699 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T00:53:38.604 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T00:53:38.637 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T00:53:38.659 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T00:53:38.660 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:38.660 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T00:53:38.660 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T00:53:38.660 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T00:53:38.660 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:38.684 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T00:53:38.705 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T00:53:38.709 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T00:53:38.710 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:38.710 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T00:53:38.710 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T00:53:38.710 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T00:53:38.710 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:38.727 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T00:53:38.727 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:38.727 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T00:53:38.727 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:38.772 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T00:53:38.796 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T00:53:38.796 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:38.796 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T00:53:38.796 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T00:53:38.796 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T00:53:38.796 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:38.934 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T00:53:38.957 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T00:53:38.958 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:38.958 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T00:53:38.958 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T00:53:38.958 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T00:53:38.958 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:39.097 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T00:53:39.120 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T00:53:39.120 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:39.120 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T00:53:39.120 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T00:53:39.120 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T00:53:39.120 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:39.402 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T00:53:39.424 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T00:53:39.424 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:39.424 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T00:53:39.424 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T00:53:39.424 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T00:53:39.425 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T00:53:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T00:53:39.644 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T00:53:39.828 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T00:53:39.878 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T00:53:39.955 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T00:53:40.043 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T00:53:40.186 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T00:53:40.228 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T00:53:40.260 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T00:53:40.288 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T00:53:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T00:53:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T00:53:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T00:53:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T00:53:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T00:53:41.866 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T00:53:41.929 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T00:53:42.892 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T00:53:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T00:53:43.021 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T00:53:43.175 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T00:53:43.206 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T00:53:43.228 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T00:53:43.228 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:43.229 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T00:53:43.229 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T00:53:43.229 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T00:53:43.229 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:43.290 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T00:53:43.301 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T00:53:43.398 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T00:53:43.446 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T00:53:43.625 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T00:53:43.667 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T00:53:43.707 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T00:53:43.740 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T00:53:43.767 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T00:53:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T00:53:43.997 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T00:53:44.029 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T00:53:44.069 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T00:53:44.099 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T00:53:44.137 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T00:53:44.346 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T00:53:44.376 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T00:53:44.399 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T00:53:44.399 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:44.399 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T00:53:44.399 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T00:53:44.399 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T00:53:44.400 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T00:53:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T00:53:44.465 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:44.465 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T00:53:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T00:53:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T00:53:44.533 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:44.533 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T00:53:44.533 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T00:53:44.533 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T00:53:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:44.689 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:53:44.726 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:53:44.746 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:53:44.836 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T00:53:44.859 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T00:53:44.859 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:44.859 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T00:53:44.859 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T00:53:44.859 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T00:53:44.859 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:53:45.571 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T00:53:45.639 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T00:53:45.680 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T00:53:45.764 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T00:53:45.793 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T00:53:45.817 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T00:53:45.843 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T00:53:45.870 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T00:53:45.870 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T00:53:45.954 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T00:53:45.978 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T00:53:45.978 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T00:53:46.063 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T00:53:46.088 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T00:53:46.094 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:53:46.094 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:53:46.282 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:53:46.282 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T00:53:46.366 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T00:53:46.391 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T00:53:46.454 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T00:53:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T00:53:49.414 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T00:53:49.456 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T00:53:49.487 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T00:53:49.514 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T00:53:49.696 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T00:53:49.744 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T00:53:49.776 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T00:53:49.809 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T00:53:49.838 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T00:53:49.885 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T00:53:50.271 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-03T00:53:50.271 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-06-03T00:53:50.271 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-06-03T00:53:50.271 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-06-03T00:53:50.271 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-06-03T00:53:50.271 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-06-03T00:53:50.271 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-06-03T00:53:50.272 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-06-03T00:53:50.272 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:50.478 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:53:50.486 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T00:53:50.516 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:53:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-03T00:53:50.533 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:50.533 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-03T00:53:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:50.536 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:53:51.361 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T00:53:51.463 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T00:53:51.504 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T00:53:51.593 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T00:53:51.626 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T00:53:51.650 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T00:53:51.676 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T00:53:51.702 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T00:53:51.702 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T00:53:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T00:53:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T00:53:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T00:53:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T00:53:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T00:53:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:53:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:53:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:53:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:52.194 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T00:53:52.195 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T00:53:52.196 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T00:53:52.196 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T00:53:52.196 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T00:53:52.196 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T00:53:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T00:53:52.203 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T00:53:52.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T00:53:52.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T00:53:52.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T00:53:52.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T00:53:52.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T00:53:52.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T00:53:52.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T00:53:52.210 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T00:53:52.211 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T00:53:52.212 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T00:53:52.213 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T00:53:52.214 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T00:53:52.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T00:53:52.216 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T00:53:52.217 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T00:53:52.218 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T00:53:52.219 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T00:53:52.220 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T00:53:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T00:53:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T00:53:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-03T00:53:52.288 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:52.289 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-03T00:53:52.289 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T00:53:52.289 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-03T00:53:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T00:53:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-03T00:53:52.648 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:52.648 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-03T00:53:52.648 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T00:53:52.648 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-03T00:53:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T00:53:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-03T00:53:53.254 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:53.254 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-03T00:53:53.254 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T00:53:53.254 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-03T00:53:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-03T00:53:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-03T00:53:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-03T00:53:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-03T00:53:53.807 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-03T00:53:53.893 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-03T00:53:54.009 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-03T00:53:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-03T00:53:54.231 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-03T00:53:54.279 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-03T00:53:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-03T00:53:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T00:53:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:54.712 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-06-03T00:53:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T00:53:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T00:53:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T00:53:54.712 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T00:53:54.713 INFO:teuthology.orchestra.run.smithi179.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.714 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.715 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T00:53:54.716 INFO:teuthology.orchestra.run.smithi179.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T00:53:54.717 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T00:53:54.718 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T00:53:54.719 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.720 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T00:53:54.721 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T00:53:54.722 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T00:53:54.723 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T00:53:54.724 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T00:53:54.725 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T00:53:54.726 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T00:53:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T00:53:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T00:53:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T00:53:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T00:53:54.727 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:53:54.727 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-06-03T00:53:54.985 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T00:53:55.023 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-03T00:53:55.101 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-03T00:53:55.186 DEBUG:teuthology.parallel:result is None 2024-06-03T00:53:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-03T00:53:55.243 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T00:53:56.526 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-03T00:53:56.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T00:53:57.676 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-03T00:53:57.704 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T00:53:57.805 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T00:53:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T00:53:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T00:53:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T00:53:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T00:53:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T00:53:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T00:53:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T00:53:57.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T00:53:57.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T00:53:57.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T00:53:57.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T00:53:57.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T00:53:57.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T00:53:57.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T00:53:57.821 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T00:53:57.822 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T00:53:57.823 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T00:53:57.824 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T00:53:57.825 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T00:53:57.826 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T00:53:57.827 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T00:53:57.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T00:53:57.829 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T00:53:57.830 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T00:53:57.831 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T00:53:57.849 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-03T00:53:58.027 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-03T00:53:58.064 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T00:53:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-03T00:53:58.131 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:58.132 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-03T00:53:58.132 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T00:53:58.132 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-03T00:53:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T00:53:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-03T00:53:58.345 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-03T00:53:58.410 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-03T00:53:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-03T00:53:59.417 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T00:53:59.514 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-03T00:53:59.514 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:59.514 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-03T00:53:59.514 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T00:53:59.514 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-03T00:53:59.515 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:59.593 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T00:53:59.691 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-03T00:53:59.691 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:59.691 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-03T00:53:59.691 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:53:59.760 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T00:53:59.835 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-03T00:53:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:53:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-03T00:53:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T00:53:59.835 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-03T00:53:59.836 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:54:00.240 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T00:54:00.311 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-03T00:54:00.312 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-03T00:54:00.312 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-03T00:54:00.312 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T00:54:00.312 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-03T00:54:00.312 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:54:00.572 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T00:54:00.572 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T00:54:00.573 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T00:54:00.576 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.577 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T00:54:00.578 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T00:54:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T00:54:00.580 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T00:54:00.581 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T00:54:00.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T00:54:00.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T00:54:00.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T00:54:00.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T00:54:00.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T00:54:00.582 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T00:54:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.584 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T00:54:00.585 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T00:54:00.586 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T00:54:00.587 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T00:54:00.587 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T00:54:00.587 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-06-03T00:54:00.587 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-06-03T00:54:01.036 DEBUG:teuthology.parallel:result is None 2024-06-03T00:54:05.267 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-03T00:54:05.324 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-03T00:54:05.379 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-03T00:54:05.441 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-03T00:54:05.480 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-03T00:54:05.761 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-03T00:54:05.821 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-03T00:54:05.854 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-03T00:54:05.895 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-03T00:54:05.942 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-03T00:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T00:54:06.841 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:54:06.893 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:54:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-03T00:54:08.260 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-03T00:54:08.375 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-03T00:54:08.439 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-03T00:54:08.583 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-03T00:54:08.619 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-03T00:54:08.651 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T00:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-03T00:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-03T00:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T00:54:08.853 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-03T00:54:08.907 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-03T00:54:08.907 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T00:54:09.027 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-03T00:54:09.066 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-03T00:54:09.100 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:54:09.100 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:54:09.429 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-03T00:54:09.429 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T00:54:09.589 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-03T00:54:09.636 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-03T00:54:09.756 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T00:54:10.012 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-03T00:54:10.013 INFO:teuthology.orchestra.run.smithi094.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-06-03T00:54:10.013 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:54:16.171 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-03T00:54:16.171 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-03T00:54:16.172 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-03T00:54:16.173 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-03T00:54:16.174 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-03T00:54:16.186 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-03T00:54:16.186 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-03T00:54:16.186 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-03T00:54:16.186 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-03T00:54:16.187 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-03T00:54:16.188 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-03T00:54:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-03T00:54:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-03T00:54:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-03T00:54:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-03T00:54:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-03T00:54:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-03T00:54:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-03T00:54:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-03T00:54:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-03T00:54:18.560 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.561 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.562 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-03T00:54:18.563 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.564 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-03T00:54:18.565 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-06-03T00:54:18.566 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.567 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-03T00:54:18.568 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-03T00:54:18.569 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-03T00:54:18.570 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.571 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-03T00:54:18.572 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-03T00:54:18.573 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-03T00:54:18.574 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-03T00:54:18.574 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:54:18.574 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-06-03T00:54:19.241 DEBUG:teuthology.parallel:result is None 2024-06-03T00:54:19.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:54:19.394 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T00:54:19.448 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T00:54:19.449 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T00:54:19.449 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T00:54:19.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:54:19.593 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T00:54:19.618 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T00:54:19.618 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T00:54:19.619 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T00:54:19.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:54:19.843 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-03T00:54:19.868 INFO:teuthology.orchestra.run.smithi179.stdout:19.0.0-2946.g3d89accf.el9 2024-06-03T00:54:19.869 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-03T00:54:19.869 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-03T00:54:19.871 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-03T00:54:19.871 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:54:19.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T00:54:19.947 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:54:19.947 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T00:54:19.977 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:54:19.977 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-03T00:54:20.008 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-03T00:54:20.009 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:54:20.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T00:54:20.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T00:54:20.184 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:54:20.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T00:54:20.224 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T00:54:20.290 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:54:20.290 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-06-03T00:54:20.318 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-03T00:54:20.385 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-03T00:54:20.385 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:54:20.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T00:54:20.451 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T00:54:20.553 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:54:20.553 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T00:54:20.578 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T00:54:20.643 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:54:20.643 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-03T00:54:20.667 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-03T00:54:20.732 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-03T00:54:20.732 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:54:20.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T00:54:20.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T00:54:20.899 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:54:20.899 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T00:54:20.922 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T00:54:20.985 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:54:20.985 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-06-03T00:54:21.009 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-03T00:54:21.071 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-03T00:54:21.161 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-03T00:54:21.161 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:54:21.161 INFO:tasks.cephadm:Cluster fsid is d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:54:21.162 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-03T00:54:21.162 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.94', 'mon.b': '172.21.15.107', 'mon.c': '172.21.15.179'} 2024-06-03T00:54:21.162 INFO:tasks.cephadm:First mon is mon.a on smithi094 2024-06-03T00:54:21.162 INFO:tasks.cephadm:First mgr is x 2024-06-03T00:54:21.162 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-03T00:54:21.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-06-03T00:54:21.226 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-06-03T00:54:21.249 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-06-03T00:54:21.273 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-03T00:54:21.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:54:21.415 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-03T00:54:21.557 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:54:21.559 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-03T00:54:21.559 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:23.334 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 00:54 /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:23.334 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:24.395 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 00:54 /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:24.395 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:24.876 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 3 00:54 /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:24.876 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:24.938 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:24.957 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-03T00:54:24.999 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-03T00:54:24.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T00:54:25.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T00:54:25.008 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-03T00:54:25.851 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T00:54:25.914 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T00:54:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T00:54:57.560 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-06-03T00:54:57.560 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T00:54:57.561 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T00:54:57.561 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-06-03T00:54:57.561 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T00:54:57.561 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-06-03T00:54:57.561 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-06-03T00:54:58.832 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-06-03T00:54:58.832 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T00:54:58.832 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T00:54:58.832 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-06-03T00:54:58.832 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T00:54:58.832 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-06-03T00:54:58.832 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-06-03T00:55:03.285 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T00:55:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T00:55:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-03T00:55:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-06-03T00:55:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-03T00:55:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T00:55:03.286 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T00:55:03.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-06-03T00:55:03.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-06-03T00:55:03.449 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-06-03T00:55:03.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-06-03T00:55:03.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-06-03T00:55:03.593 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-06-03T00:55:03.621 INFO:tasks.cephadm:Writing seed config... 2024-06-03T00:55:03.622 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-03T00:55:03.622 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-03T00:55:03.622 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-03T00:55:03.623 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-03T00:55:03.624 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-03T00:55:03.625 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-03T00:55:03.626 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-03T00:55:03.626 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-06-03T00:55:03.626 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:55:03.626 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-03T00:55:03.676 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d037ccd0-2143-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-03T00:55:03.677 DEBUG:teuthology.orchestra.run.smithi094:mon.a> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a.service 2024-06-03T00:55:03.724 DEBUG:teuthology.orchestra.run.smithi094:mgr.x> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-03T00:55:03.770 INFO:tasks.cephadm:Bootstrapping... 2024-06-03T00:55:03.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-03T00:55:04.043 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-06-03T00:55:04.043 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', 'd037ccd0-2143-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-06-03T00:55:04.044 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-03T00:55:04.044 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-06-03T00:55:04.101 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 5.0.2 2024-06-03T00:55:04.102 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-06-03T00:55:04.102 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-06-03T00:55:04.118 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T00:55:04.118 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T00:55:04.134 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T00:55:04.134 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-06-03T00:55:04.153 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-06-03T00:55:04.171 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-06-03T00:55:04.171 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-06-03T00:55:04.172 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-06-03T00:55:04.231 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 5.0.2 2024-06-03T00:55:04.232 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-03T00:55:04.232 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-06-03T00:55:04.232 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-06-03T00:55:04.246 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-03T00:55:04.247 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-03T00:55:04.262 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-03T00:55:04.262 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-06-03T00:55:04.278 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-06-03T00:55:04.295 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-06-03T00:55:04.295 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-06-03T00:55:04.295 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-06-03T00:55:04.295 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:04.296 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 139647568856448 on /run/cephadm/d037ccd0-2143-11ef-bc9b-c7b262605968.lock 2024-06-03T00:55:04.296 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 139647568856448 acquired on /run/cephadm/d037ccd0-2143-11ef-bc9b-c7b262605968.lock 2024-06-03T00:55:04.296 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-06-03T00:55:04.297 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-06-03T00:55:04.297 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-06-03T00:55:04.307 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-06-03T00:55:04.308 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-06-03T00:55:04.316 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-03T00:55:04.316 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-03T00:55:04.316 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1781sec hoplimit 64 pref medium 2024-06-03T00:55:04.324 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-03T00:55:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-03T00:55:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T00:55:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-03T00:55:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-06-03T00:55:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-03T00:55:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-06-03T00:55:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-06-03T00:55:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-03T00:55:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-03T00:55:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T00:55:14.794 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-03T00:55:14.794 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-03T00:55:14.795 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-06-03T00:55:14.795 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-03T00:55:14.795 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-03T00:55:14.795 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-03T00:55:14.795 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-03T00:55:16.037 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-03T00:55:16.037 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-03T00:55:16.037 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-06-03T00:55:17.215 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-06-03T00:55:17.216 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-06-03T00:55:18.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQB1FF1mkx+IKBAAdlqc2zj1SQyb6kUfO5phug== 2024-06-03T00:55:19.729 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQB2FF1mmtfHMxAAjMfnowZxOTPkY3kBbGNcHg== 2024-06-03T00:55:21.110 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQB4FF1m9x8SCRAA7hVhwOAimuMzt8ww61DxJg== 2024-06-03T00:55:21.113 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-06-03T00:55:22.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T00:55:22.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for a [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:55:22.380 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-06-03T00:55:23.534 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.a on 2024-06-03T00:55:23.858 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-03T00:55:24.098 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-03T00:55:24.422 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968.target. 2024-06-03T00:55:24.422 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968.target. 2024-06-03T00:55:24.806 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a 2024-06-03T00:55:24.807 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-03T00:55:25.103 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968.target.wants/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@.service. 2024-06-03T00:55:25.770 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:25 smithi094 podman[31733]: 2024-06-03 00:55:25.517356561 +0000 UTC m=+0.140571027 container create 0d313fbdc39f206ed2e6c05ffaa5534a61b5d597823cac44297b73d9629b209b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T00:55:25.770 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:25 smithi094 podman[31733]: 2024-06-03 00:55:25.768696928 +0000 UTC m=+0.391911446 container init 0d313fbdc39f206ed2e6c05ffaa5534a61b5d597823cac44297b73d9629b209b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5) 2024-06-03T00:55:25.928 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T00:55:25.928 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-06-03T00:55:25.947 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T00:55:25.947 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-06-03T00:55:25.947 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-06-03T00:55:25.947 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T00:55:25.948 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-06-03T00:55:25.948 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-06-03T00:55:26.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:25 smithi094 podman[31733]: 2024-06-03 00:55:25.779459631 +0000 UTC m=+0.402674113 container start 0d313fbdc39f206ed2e6c05ffaa5534a61b5d597823cac44297b73d9629b209b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T00:55:26.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:25 smithi094 ceph-mon[31751]: mkfs d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:26.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:25 smithi094 ceph-mon[31751]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T00:55:26.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:25 smithi094 bash[31733]: 0d313fbdc39f206ed2e6c05ffaa5534a61b5d597823cac44297b73d9629b209b 2024-06-03T00:55:26.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:25 smithi094 systemd[1]: Started Ceph mon.a for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:55:27.026 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: monmap epoch 1 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: last_changed 2024-06-03T00:55:21.644119+0000 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: min_mon_release 19 (squid) 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: election_strategy: 1 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: fsmap 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: mgrmap e1: no daemons active 2024-06-03T00:55:27.027 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:26 smithi094 ceph-mon[31751]: from='client.? 172.21.15.94:0/973035851' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.802813s) 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-06-03T00:55:27.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.566+0000 7fc924200700 1 Processor -- start 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.566+0000 7fc924200700 1 -- start start 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.567+0000 7fc924200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.567+0000 7fc924200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c07be70 con 0x7fc91c106690 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.568+0000 7fc921800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.568+0000 7fc921800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c106ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42612/0 (socket says 172.21.15.94:42612) 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.568+0000 7fc921800700 1 -- 172.21.15.94:0/3391348771 learned_addr learned my addr 172.21.15.94:0/3391348771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:27.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.568+0000 7fc921800700 1 -- 172.21.15.94:0/3391348771 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91c106ff0 con 0x7fc91c106690 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.568+0000 7fc921800700 1 --2- 172.21.15.94:0/3391348771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c106ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc9100098a0 tx=0x7fc910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc62986de8832839 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.569+0000 7fc91be00700 1 -- 172.21.15.94:0/3391348771 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc910004160 con 0x7fc91c106690 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.569+0000 7fc91be00700 1 -- 172.21.15.94:0/3391348771 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910004300 con 0x7fc91c106690 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.569+0000 7fc91be00700 1 -- 172.21.15.94:0/3391348771 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9100036a0 con 0x7fc91c106690 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.569+0000 7fc924200700 1 -- 172.21.15.94:0/3391348771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 msgr2=0x7fc91c106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.569+0000 7fc924200700 1 --2- 172.21.15.94:0/3391348771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c106ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc9100098a0 tx=0x7fc910009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.569+0000 7fc924200700 1 -- 172.21.15.94:0/3391348771 shutdown_connections 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.570+0000 7fc924200700 1 --2- 172.21.15.94:0/3391348771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c106ab0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.570+0000 7fc924200700 1 -- 172.21.15.94:0/3391348771 >> 172.21.15.94:0/3391348771 conn(0x7fc91c1022d0 msgr2=0x7fc91c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.570+0000 7fc924200700 1 -- 172.21.15.94:0/3391348771 shutdown_connections 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.570+0000 7fc924200700 1 -- 172.21.15.94:0/3391348771 wait complete. 2024-06-03T00:55:27.353 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.571+0000 7fc924200700 1 Processor -- start 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.571+0000 7fc924200700 1 -- start start 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.571+0000 7fc924200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.571+0000 7fc924200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c107520 con 0x7fc91c106690 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.572+0000 7fc921800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.572+0000 7fc921800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42626/0 (socket says 172.21.15.94:42626) 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.572+0000 7fc921800700 1 -- 172.21.15.94:0/973035851 learned_addr learned my addr 172.21.15.94:0/973035851 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.572+0000 7fc921800700 1 -- 172.21.15.94:0/973035851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91c19dc30 con 0x7fc91c106690 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.573+0000 7fc921800700 1 --2- 172.21.15.94:0/973035851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c19d6f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc910000c00 tx=0x7fc910004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.573+0000 7fc91a000700 1 -- 172.21.15.94:0/973035851 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc910003db0 con 0x7fc91c106690 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.573+0000 7fc91a000700 1 -- 172.21.15.94:0/973035851 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910011630 con 0x7fc91c106690 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.573+0000 7fc91a000700 1 -- 172.21.15.94:0/973035851 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc910011950 con 0x7fc91c106690 2024-06-03T00:55:27.354 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.573+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91c19dec0 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.573+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91c1a0b60 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.574+0000 7fc91a000700 1 -- 172.21.15.94:0/973035851 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc910011af0 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.574+0000 7fc91a000700 1 -- 172.21.15.94:0/973035851 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc91001c450 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.574+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91c063af0 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.578+0000 7fc91a000700 1 -- 172.21.15.94:0/973035851 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc910032020 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.656+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc91c1a1050 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.657+0000 7fc91a000700 1 -- 172.21.15.94:0/973035851 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc910004260 con 0x7fc91c106690 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.658+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 msgr2=0x7fc91c19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.658+0000 7fc924200700 1 --2- 172.21.15.94:0/973035851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c19d6f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc910000c00 tx=0x7fc910004750 comp rx=0 tx=0).stop 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.658+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 shutdown_connections 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.658+0000 7fc924200700 1 --2- 172.21.15.94:0/973035851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc91c106690 0x7fc91c19d6f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.658+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 >> 172.21.15.94:0/973035851 conn(0x7fc91c1022d0 msgr2=0x7fc91c193d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:27.355 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.658+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 shutdown_connections 2024-06-03T00:55:27.356 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:26.658+0000 7fc924200700 1 -- 172.21.15.94:0/973035851 wait complete. 2024-06-03T00:55:27.356 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-06-03T00:55:27.356 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-06-03T00:55:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:28 smithi094 ceph-mon[31751]: from='client.? 172.21.15.94:0/1897595849' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T00:55:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:28 smithi094 ceph-mon[31751]: from='client.? 172.21.15.94:0/1897595849' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-03T00:55:28.712 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T00:55:28.713 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:28.714 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T00:55:28.714 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T00:55:28.714 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:28.714 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T00:55:28.714 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T00:55:28.714 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.894+0000 7f31d7000700 1 Processor -- start 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.894+0000 7f31d7000700 1 -- start start 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.895+0000 7f31d7000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.895+0000 7f31d7000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31d007c890 con 0x7f31d007de60 2024-06-03T00:55:28.715 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.895+0000 7f31cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:28.716 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.895+0000 7f31cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42630/0 (socket says 172.21.15.94:42630) 2024-06-03T00:55:28.716 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.895+0000 7f31cfe00700 1 -- 172.21.15.94:0/2413667213 learned_addr learned my addr 172.21.15.94:0/2413667213 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:28.716 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.896+0000 7f31cfe00700 1 -- 172.21.15.94:0/2413667213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d007ca10 con 0x7f31d007de60 2024-06-03T00:55:28.716 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.896+0000 7f31cfe00700 1 --2- 172.21.15.94:0/2413667213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d007c2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f31c00098a0 tx=0x7f31c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5975d60f2bf2ef server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:28.716 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.896+0000 7f31cf400700 1 -- 172.21.15.94:0/2413667213 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c0004160 con 0x7f31d007de60 2024-06-03T00:55:28.716 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31cf400700 1 -- 172.21.15.94:0/2413667213 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c0004300 con 0x7f31d007de60 2024-06-03T00:55:28.716 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31cf400700 1 -- 172.21.15.94:0/2413667213 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c00036a0 con 0x7f31d007de60 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31d7000700 1 -- 172.21.15.94:0/2413667213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 msgr2=0x7f31d007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31d7000700 1 --2- 172.21.15.94:0/2413667213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d007c2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f31c00098a0 tx=0x7f31c0009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31d7000700 1 -- 172.21.15.94:0/2413667213 shutdown_connections 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31d7000700 1 --2- 172.21.15.94:0/2413667213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d007c2c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31d7000700 1 -- 172.21.15.94:0/2413667213 >> 172.21.15.94:0/2413667213 conn(0x7f31d01033e0 msgr2=0x7f31d0105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31d7000700 1 -- 172.21.15.94:0/2413667213 shutdown_connections 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.897+0000 7f31d7000700 1 -- 172.21.15.94:0/2413667213 wait complete. 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.898+0000 7f31d7000700 1 Processor -- start 2024-06-03T00:55:28.717 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.899+0000 7f31d7000700 1 -- start start 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.899+0000 7f31d7000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.899+0000 7f31d7000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31d007cf40 con 0x7f31d007de60 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.899+0000 7f31cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.899+0000 7f31cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42634/0 (socket says 172.21.15.94:42634) 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.899+0000 7f31cfe00700 1 -- 172.21.15.94:0/1897595849 learned_addr learned my addr 172.21.15.94:0/1897595849 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.899+0000 7f31cfe00700 1 -- 172.21.15.94:0/1897595849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d019dc30 con 0x7f31d007de60 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31cfe00700 1 --2- 172.21.15.94:0/1897595849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d019d6f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31c0000c00 tx=0x7f31c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:28.718 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31cd600700 1 -- 172.21.15.94:0/1897595849 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c0003db0 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31cd600700 1 -- 172.21.15.94:0/1897595849 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c0011630 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d019dec0 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31cd600700 1 -- 172.21.15.94:0/1897595849 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31c0011950 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d01a0bb0 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31cd600700 1 -- 172.21.15.94:0/1897595849 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31c0011af0 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.900+0000 7f31cd600700 1 -- 172.21.15.94:0/1897595849 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f31c001c450 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.901+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d0063af0 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.904+0000 7f31cd600700 1 -- 172.21.15.94:0/1897595849 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f31c0032020 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.980+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f31d002deb0 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.991+0000 7f31cd600700 1 -- 172.21.15.94:0/1897595849 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f31c0004260 con 0x7f31d007de60 2024-06-03T00:55:28.719 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.993+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 msgr2=0x7f31d019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.993+0000 7f31d7000700 1 --2- 172.21.15.94:0/1897595849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d019d6f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31c0000c00 tx=0x7f31c0004750 comp rx=0 tx=0).stop 2024-06-03T00:55:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.993+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 shutdown_connections 2024-06-03T00:55:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.993+0000 7f31d7000700 1 --2- 172.21.15.94:0/1897595849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31d007de60 0x7f31d019d6f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.993+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 >> 172.21.15.94:0/1897595849 conn(0x7f31d01033e0 msgr2=0x7f31d01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.993+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 shutdown_connections 2024-06-03T00:55:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:27.993+0000 7f31d7000700 1 -- 172.21.15.94:0/1897595849 wait complete. 2024-06-03T00:55:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-06-03T00:55:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:29 smithi094 ceph-mon[31751]: from='client.? 172.21.15.94:0/3242002327' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:55:29.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.268+0000 7f29f2600700 1 Processor -- start 2024-06-03T00:55:29.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29f2600700 1 -- start start 2024-06-03T00:55:29.964 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29f2600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec07cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:29.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29f2600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec07d070 con 0x7f29ec07e680 2024-06-03T00:55:29.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29eb400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec07cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29eb400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec07cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42644/0 (socket says 172.21.15.94:42644) 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29eb400700 1 -- 172.21.15.94:0/4291571428 learned_addr learned my addr 172.21.15.94:0/4291571428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29eb400700 1 -- 172.21.15.94:0/4291571428 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec07d1f0 con 0x7f29ec07e680 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.269+0000 7f29eb400700 1 --2- 172.21.15.94:0/4291571428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec07cb30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e80 tx=0x7f29dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4aefc4445e416c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.270+0000 7f29ea000700 1 -- 172.21.15.94:0/4291571428 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc017070 con 0x7f29ec07e680 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.270+0000 7f29ea000700 1 -- 172.21.15.94:0/4291571428 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29dc00b8c0 con 0x7f29ec07e680 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.270+0000 7f29ea000700 1 -- 172.21.15.94:0/4291571428 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc0045c0 con 0x7f29ec07e680 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.271+0000 7f29f2600700 1 -- 172.21.15.94:0/4291571428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 msgr2=0x7f29ec07cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.271+0000 7f29f2600700 1 --2- 172.21.15.94:0/4291571428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec07cb30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e80 tx=0x7f29dc00b330 comp rx=0 tx=0).stop 2024-06-03T00:55:29.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.271+0000 7f29f2600700 1 -- 172.21.15.94:0/4291571428 shutdown_connections 2024-06-03T00:55:29.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.271+0000 7f29f2600700 1 --2- 172.21.15.94:0/4291571428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec07cb30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.271+0000 7f29f2600700 1 -- 172.21.15.94:0/4291571428 >> 172.21.15.94:0/4291571428 conn(0x7f29ec103420 msgr2=0x7f29ec105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.271+0000 7f29f2600700 1 -- 172.21.15.94:0/4291571428 shutdown_connections 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.271+0000 7f29f2600700 1 -- 172.21.15.94:0/4291571428 wait complete. 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.272+0000 7f29f2600700 1 Processor -- start 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.272+0000 7f29f2600700 1 -- start start 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.272+0000 7f29f2600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.272+0000 7f29f2600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec10e3f0 con 0x7f29ec07e680 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.273+0000 7f29eb400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.273+0000 7f29eb400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42652/0 (socket says 172.21.15.94:42652) 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.273+0000 7f29eb400700 1 -- 172.21.15.94:0/3242002327 learned_addr learned my addr 172.21.15.94:0/3242002327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.273+0000 7f29eb400700 1 -- 172.21.15.94:0/3242002327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec19e430 con 0x7f29ec07e680 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.273+0000 7f29eb400700 1 --2- 172.21.15.94:0/3242002327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec19def0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29dc004350 tx=0x7f29dc00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:29.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.273+0000 7f29e3e00700 1 -- 172.21.15.94:0/3242002327 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc020020 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.274+0000 7f29e3e00700 1 -- 172.21.15.94:0/3242002327 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29dc012430 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.274+0000 7f29e3e00700 1 -- 172.21.15.94:0/3242002327 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29dc017040 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.274+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ec19e6c0 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.274+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ec1a13b0 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.274+0000 7f29e3e00700 1 -- 172.21.15.94:0/3242002327 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f29dc023020 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.274+0000 7f29e3e00700 1 -- 172.21.15.94:0/3242002327 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f29dc00f050 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.275+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ec063af0 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.278+0000 7f29e3e00700 1 -- 172.21.15.94:0/3242002327 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f29dc02a020 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.354+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f29ec1a1580 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.355+0000 7f29e3e00700 1 -- 172.21.15.94:0/3242002327 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f29dc00b860 con 0x7f29ec07e680 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.356+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 msgr2=0x7f29ec19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.356+0000 7f29f2600700 1 --2- 172.21.15.94:0/3242002327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec19def0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29dc004350 tx=0x7f29dc00bf80 comp rx=0 tx=0).stop 2024-06-03T00:55:29.972 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.356+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 shutdown_connections 2024-06-03T00:55:29.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.356+0000 7f29f2600700 1 --2- 172.21.15.94:0/3242002327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ec07e680 0x7f29ec19def0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:29.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.356+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 >> 172.21.15.94:0/3242002327 conn(0x7f29ec103420 msgr2=0x7f29ec103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:29.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.357+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 shutdown_connections 2024-06-03T00:55:29.973 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:29.357+0000 7f29f2600700 1 -- 172.21.15.94:0/3242002327 wait complete. 2024-06-03T00:55:29.973 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-06-03T00:55:30.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:30 smithi094 systemd[1]: Stopping Ceph mon.a for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T00:55:30.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a[31747]: 2024-06-03T00:55:30.453+0000 7f0f28200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T00:55:30.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a[31747]: 2024-06-03T00:55:30.453+0000 7f0f28200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-03T00:55:30.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:30 smithi094 podman[31994]: 2024-06-03 00:55:30.577867309 +0000 UTC m=+0.288413047 container died 0d313fbdc39f206ed2e6c05ffaa5534a61b5d597823cac44297b73d9629b209b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-03T00:55:31.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:31 smithi094 podman[31994]: 2024-06-03 00:55:31.588146472 +0000 UTC m=+1.298692215 container cleanup 0d313fbdc39f206ed2e6c05ffaa5534a61b5d597823cac44297b73d9629b209b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-03T00:55:31.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:31 smithi094 bash[31994]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a 2024-06-03T00:55:32.087 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:31 smithi094 podman[32006]: 2024-06-03 00:55:31.863846757 +0000 UTC m=+1.263607144 container remove 0d313fbdc39f206ed2e6c05ffaa5534a61b5d597823cac44297b73d9629b209b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T00:55:32.487 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-03T00:55:32.487 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 systemd[1]: Stopped Ceph mon.a for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:55:32.487 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a.service: Consumed 1.708s CPU time. 2024-06-03T00:55:32.487 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 systemd[1]: Starting Ceph mon.a for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T00:55:32.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 podman[32100]: 2024-06-03 00:55:32.391611819 +0000 UTC m=+0.031538386 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-03T00:55:32.751 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 podman[32100]: 2024-06-03 00:55:32.497944588 +0000 UTC m=+0.137871191 container create a6a853b7a1fe6e4bd37b84b1e5232351ffd065385417ce80dd4bd9969b82c816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-06-03T00:55:32.836 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-03T00:55:33.052 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 podman[32100]: 2024-06-03 00:55:32.751170839 +0000 UTC m=+0.391097486 container init a6a853b7a1fe6e4bd37b84b1e5232351ffd065385417ce80dd4bd9969b82c816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-03T00:55:33.052 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 podman[32100]: 2024-06-03 00:55:32.761267959 +0000 UTC m=+0.401194568 container start a6a853b7a1fe6e4bd37b84b1e5232351ffd065385417ce80dd4bd9969b82c816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-03T00:55:33.052 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: set uid:gid to 167:167 (ceph:ceph) 2024-06-03T00:55:33.052 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-03T00:55:33.052 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: pidfile_write: ignore empty --pid-file 2024-06-03T00:55:33.052 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: load: jerasure load: lrc 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: RocksDB version: 7.9.2 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Git sha 0 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: DB SUMMARY 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: DB Session ID: 782Z7T2GND35JHP98A0F 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: CURRENT file: CURRENT 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: IDENTITY file: IDENTITY 2024-06-03T00:55:33.053 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104152 ; 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.error_if_exists: 0 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.create_if_missing: 0 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.paranoid_checks: 1 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.env: 0x55c120a8ac60 2024-06-03T00:55:33.054 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.fs: PosixFileSystem 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.info_log: 0x55c156a05b60 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_file_opening_threads: 16 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.statistics: (nil) 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.use_fsync: 0 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_log_file_size: 0 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.keep_log_file_num: 1000 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.recycle_log_file_num: 0 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.allow_fallocate: 1 2024-06-03T00:55:33.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.allow_mmap_reads: 0 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.allow_mmap_writes: 0 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.use_direct_reads: 0 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.create_missing_column_families: 0 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.db_log_dir: 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.wal_dir: 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-03T00:55:33.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.advise_random_on_open: 1 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.db_write_buffer_size: 0 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.write_buffer_manager: 0x55c1569d61e0 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.rate_limiter: (nil) 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.wal_recovery_mode: 2 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.enable_thread_tracking: 0 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.enable_pipelined_write: 0 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.unordered_write: 0 2024-06-03T00:55:33.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-03T00:55:33.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-03T00:55:33.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-03T00:55:33.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-03T00:55:33.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.row_cache: None 2024-06-03T00:55:33.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.wal_filter: None 2024-06-03T00:55:33.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-03T00:55:33.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.allow_ingest_behind: 0 2024-06-03T00:55:33.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.two_write_queues: 0 2024-06-03T00:55:33.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.manual_wal_flush: 0 2024-06-03T00:55:33.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.wal_compression: 0 2024-06-03T00:55:33.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.atomic_flush: 0 2024-06-03T00:55:33.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-03T00:55:33.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.log_readahead_size: 0 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.best_efforts_recovery: 0 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.allow_data_in_errors: 0 2024-06-03T00:55:33.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.db_host_id: __hostname__ 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_background_jobs: 2 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_background_compactions: -1 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_subcompactions: 1 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_total_wal_size: 0 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-03T00:55:33.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_open_files: -1 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bytes_per_sync: 0 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_readahead_size: 0 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_background_flushes: -1 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Compression algorithms supported: 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kZSTD supported: 0 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kXpressCompression supported: 0 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kLZ4HCCompression supported: 1 2024-06-03T00:55:33.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kZlibCompression supported: 1 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kSnappyCompression supported: 1 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kLZ4Compression supported: 1 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: kBZip2Compression supported: 0 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.merge_operator: 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_filter: None 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_filter_factory: None 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.sst_partitioner_factory: None 2024-06-03T00:55:33.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c156a05c80) 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_type: 0 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_index_type: 0 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_shortening: 1 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-03T00:55:33.065 INFO:journalctl@ceph.mon.a.smithi094.stdout: checksum: 4 2024-06-03T00:55:33.066 INFO:journalctl@ceph.mon.a.smithi094.stdout: no_block_cache: 0 2024-06-03T00:55:33.066 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache: 0x55c156a07090 2024-06-03T00:55:33.066 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-06-03T00:55:33.066 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_options: 2024-06-03T00:55:33.066 INFO:journalctl@ceph.mon.a.smithi094.stdout: capacity : 536870912 2024-06-03T00:55:33.066 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_shard_bits : 4 2024-06-03T00:55:33.066 INFO:journalctl@ceph.mon.a.smithi094.stdout: strict_capacity_limit : 0 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_compressed: (nil) 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: persistent_cache: (nil) 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size: 4096 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size_deviation: 10 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_restart_interval: 16 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_block_restart_interval: 1 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: metadata_block_size: 4096 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: partition_filters: 0 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: use_delta_encoding: 1 2024-06-03T00:55:33.067 INFO:journalctl@ceph.mon.a.smithi094.stdout: filter_policy: bloomfilter 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: whole_key_filtering: 1 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: verify_compression: 0 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: format_version: 5 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_index_compression: 1 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_align: 0 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: max_auto_readahead_size: 262144 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: prepopulate_block_cache: 0 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: initial_auto_readahead_size: 8192 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.write_buffer_size: 33554432 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_write_buffer_number: 2 2024-06-03T00:55:33.068 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression: NoCompression 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression: Disabled 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.prefix_extractor: nullptr 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.num_levels: 7 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 bash[32100]: a6a853b7a1fe6e4bd37b84b1e5232351ffd065385417ce80dd4bd9969b82c816 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-03T00:55:33.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.level: 32767 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.strategy: 0 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-03T00:55:33.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.enabled: false 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.target_file_size_base: 67108864 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-03T00:55:33.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.arena_block_size: 1048576 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-03T00:55:33.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.disable_auto_compactions: 0 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.table_properties_collectors: 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.inplace_update_support: 0 2024-06-03T00:55:33.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.bloom_locality: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.max_successive_merges: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.paranoid_file_checks: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.force_consistency_checks: 1 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.report_bg_io_stats: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.ttl: 2592000 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-03T00:55:33.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.enable_blob_files: false 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.min_blob_size: 0 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.blob_file_size: 268435456 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.blob_file_starting_level: 0 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-03T00:55:33.075 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3b8df813-001b-4975-a750-5ca920f53dbb 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717376132821692, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717376132824452, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99357, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97277, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14034, "raw_average_key_size": 49, "raw_value_size": 89613, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717376132, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3b8df813-001b-4975-a750-5ca920f53dbb", "db_session_id": "782Z7T2GND35JHP98A0F", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717376132824671, "job": 1, "event": "recovery_finished"} 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 systemd[1]: Started Ceph mon.a for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c156af6000 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: DB pointer 0x55c156ae0000 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-03T00:55:33.076 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** DB Stats ** 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: L0 2/0 98.86 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T00:55:33.077 INFO:journalctl@ceph.mon.a.smithi094.stdout: Sum 2/0 98.86 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-03T00:55:33.078 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative compaction: 0.00 GB write, 5.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval compaction: 0.00 GB write, 5.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache BinnedLRUCache@0x55c156a07090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.9e-05 secs_since: 0 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: starting mon.a rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???) e1 preinit fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???).mds e1 new map 2024-06-03T00:55:33.079 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???).mds e1 print_map 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout: e1 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout: legacy client fscid: -1 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout: No filesystems configured 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-03T00:55:33.080 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: monmap epoch 1 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: last_changed 2024-06-03T00:55:21.644119+0000 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: min_mon_release 19 (squid) 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: election_strategy: 1 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: fsmap 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: osdmap e1: 0 total, 0 up, 0 in 2024-06-03T00:55:33.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:32 smithi094 ceph-mon[32119]: mgrmap e1: no daemons active 2024-06-03T00:55:34.292 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.560+0000 7f72d9000700 1 Processor -- start 2024-06-03T00:55:34.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.560+0000 7f72d9000700 1 -- start start 2024-06-03T00:55:34.296 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.561+0000 7f72d9000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d41070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.561+0000 7f72d9000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41076b0 con 0x7f72d4106cc0 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.561+0000 7f72d1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d41070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.562+0000 7f72d1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d41070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42664/0 (socket says 172.21.15.94:42664) 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.562+0000 7f72d1e00700 1 -- 172.21.15.94:0/3651272524 learned_addr learned my addr 172.21.15.94:0/3651272524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.562+0000 7f72d1e00700 1 -- 172.21.15.94:0/3651272524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d4107ec0 con 0x7f72d4106cc0 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.562+0000 7f72d1e00700 1 --2- 172.21.15.94:0/3651272524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d41070e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f72c40098a0 tx=0x7f72c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42400bb1ec80805c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.563+0000 7f72d0a00700 1 -- 172.21.15.94:0/3651272524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72c4004160 con 0x7f72d4106cc0 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.563+0000 7f72d0a00700 1 -- 172.21.15.94:0/3651272524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f72c4004300 con 0x7f72d4106cc0 2024-06-03T00:55:34.297 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.563+0000 7f72d0a00700 1 -- 172.21.15.94:0/3651272524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72c4011b50 con 0x7f72d4106cc0 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.563+0000 7f72d9000700 1 -- 172.21.15.94:0/3651272524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 msgr2=0x7f72d41070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.563+0000 7f72d9000700 1 --2- 172.21.15.94:0/3651272524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d41070e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f72c40098a0 tx=0x7f72c4009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.564+0000 7f72d9000700 1 -- 172.21.15.94:0/3651272524 shutdown_connections 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.564+0000 7f72d9000700 1 --2- 172.21.15.94:0/3651272524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d41070e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.564+0000 7f72d9000700 1 -- 172.21.15.94:0/3651272524 >> 172.21.15.94:0/3651272524 conn(0x7f72d41022c0 msgr2=0x7f72d41046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.564+0000 7f72d9000700 1 -- 172.21.15.94:0/3651272524 shutdown_connections 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.564+0000 7f72d9000700 1 -- 172.21.15.94:0/3651272524 wait complete. 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.565+0000 7f72d9000700 1 Processor -- start 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.565+0000 7f72d9000700 1 -- start start 2024-06-03T00:55:34.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.565+0000 7f72d9000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.565+0000 7f72d9000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72d410b270 con 0x7f72d4106cc0 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.566+0000 7f72d1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.566+0000 7f72d1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42678/0 (socket says 172.21.15.94:42678) 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.566+0000 7f72d1e00700 1 -- 172.21.15.94:0/270932004 learned_addr learned my addr 172.21.15.94:0/270932004 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.566+0000 7f72d1e00700 1 -- 172.21.15.94:0/270932004 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d419e4e0 con 0x7f72d4106cc0 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.566+0000 7f72d1e00700 1 --2- 172.21.15.94:0/270932004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d419dfa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f72c4000c00 tx=0x7f72c4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.567+0000 7f72caa00700 1 -- 172.21.15.94:0/270932004 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72c4003b20 con 0x7f72d4106cc0 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.567+0000 7f72caa00700 1 -- 172.21.15.94:0/270932004 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f72c4003cc0 con 0x7f72d4106cc0 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.567+0000 7f72caa00700 1 -- 172.21.15.94:0/270932004 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72c4024910 con 0x7f72d4106cc0 2024-06-03T00:55:34.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.567+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d419e770 con 0x7f72d4106cc0 2024-06-03T00:55:34.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.567+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d41a1460 con 0x7f72d4106cc0 2024-06-03T00:55:34.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.567+0000 7f72caa00700 1 -- 172.21.15.94:0/270932004 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f72c4022020 con 0x7f72d4106cc0 2024-06-03T00:55:34.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.567+0000 7f72caa00700 1 -- 172.21.15.94:0/270932004 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f72c4011e00 con 0x7f72d4106cc0 2024-06-03T00:55:34.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.568+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d4063af0 con 0x7f72d4106cc0 2024-06-03T00:55:34.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.572+0000 7f72caa00700 1 -- 172.21.15.94:0/270932004 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f72c401acd0 con 0x7f72d4106cc0 2024-06-03T00:55:34.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.648+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f72d402dca0 con 0x7f72d4106cc0 2024-06-03T00:55:34.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.651+0000 7f72caa00700 1 -- 172.21.15.94:0/270932004 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f72c4032030 con 0x7f72d4106cc0 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.653+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 msgr2=0x7f72d419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.653+0000 7f72d9000700 1 --2- 172.21.15.94:0/270932004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d419dfa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f72c4000c00 tx=0x7f72c4011c10 comp rx=0 tx=0).stop 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.653+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 shutdown_connections 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.653+0000 7f72d9000700 1 --2- 172.21.15.94:0/270932004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72d4106cc0 0x7f72d419dfa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.653+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 >> 172.21.15.94:0/270932004 conn(0x7f72d41022c0 msgr2=0x7f72d4103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.654+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 shutdown_connections 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:33.654+0000 7f72d9000700 1 -- 172.21.15.94:0/270932004 wait complete. 2024-06-03T00:55:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-03T00:55:34.321 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-03T00:55:34.322 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-06-03T00:55:34.322 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-03T00:55:34.323 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-03T00:55:34.728 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.x 2024-06-03T00:55:34.728 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-03T00:55:34.978 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968.target.wants/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@.service. 2024-06-03T00:55:34.986 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:34 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/270932004' entity='client.admin' 2024-06-03T00:55:35.631 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:35 smithi094 podman[32343]: 2024-06-03 00:55:35.620308921 +0000 UTC m=+0.373729195 container init 0dcafd448720e14b35b84e142492819c43f3f6b5408ffbc7c94ccd0d4c32d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T00:55:35.631 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:35 smithi094 podman[32343]: 2024-06-03 00:55:35.63044432 +0000 UTC m=+0.383864595 container start 0dcafd448720e14b35b84e142492819c43f3f6b5408ffbc7c94ccd0d4c32d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-03T00:55:35.741 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T00:55:35.741 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-06-03T00:55:35.757 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T00:55:35.757 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-06-03T00:55:35.758 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-06-03T00:55:35.758 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-06-03T00:55:35.780 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T00:55:35.780 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-06-03T00:55:35.800 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T00:55:35.800 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-06-03T00:55:35.801 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-06-03T00:55:35.801 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-03T00:55:35.801 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-06-03T00:55:35.802 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-06-03T00:55:35.953 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:35 smithi094 bash[32343]: 0dcafd448720e14b35b84e142492819c43f3f6b5408ffbc7c94ccd0d4c32d8ab 2024-06-03T00:55:35.953 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:35 smithi094 systemd[1]: Started Ceph mgr.x for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:55:35.953 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:35.865+0000 7fc9a95d7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T00:55:36.217 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:36.122+0000 7fc9a95d7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:55:36.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:36.259+0000 7fc9a95d7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T00:55:36.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:36.661+0000 7fc9a95d7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T00:55:36.873 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:36.779+0000 7fc9a95d7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T00:55:37.209 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:36 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/212793687' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T00:55:37.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:37.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T00:55:37.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:37.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T00:55:37.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T00:55:37.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T00:55:37.529 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T00:55:37.529 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T00:55:37.530 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:37.531 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T00:55:25.857646+0000", 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.777+0000 7f63dcc00700 1 Processor -- start 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.777+0000 7f63dcc00700 1 -- start start 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.777+0000 7f63dcc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d8108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.777+0000 7f63dcc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63d81087d0 con 0x7f63d8107de0 2024-06-03T00:55:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.778+0000 7f63d5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d8108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.778+0000 7f63d5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d8108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50748/0 (socket says 172.21.15.94:50748) 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.778+0000 7f63d5e00700 1 -- 172.21.15.94:0/721481926 learned_addr learned my addr 172.21.15.94:0/721481926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.779+0000 7f63d5e00700 1 -- 172.21.15.94:0/721481926 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d8108fe0 con 0x7f63d8107de0 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.779+0000 7f63d5e00700 1 --2- 172.21.15.94:0/721481926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d8108200 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63c80098a0 tx=0x7f63c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5647812b8bc6f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.779+0000 7f63d4a00700 1 -- 172.21.15.94:0/721481926 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63c8004160 con 0x7f63d8107de0 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.779+0000 7f63d4a00700 1 -- 172.21.15.94:0/721481926 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63c8004300 con 0x7f63d8107de0 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.779+0000 7f63d4a00700 1 -- 172.21.15.94:0/721481926 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63c8011b50 con 0x7f63d8107de0 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.780+0000 7f63dcc00700 1 -- 172.21.15.94:0/721481926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 msgr2=0x7f63d8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.780+0000 7f63dcc00700 1 --2- 172.21.15.94:0/721481926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d8108200 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63c80098a0 tx=0x7f63c8009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.780+0000 7f63dcc00700 1 -- 172.21.15.94:0/721481926 shutdown_connections 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.780+0000 7f63dcc00700 1 --2- 172.21.15.94:0/721481926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d8108200 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:37.533 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.780+0000 7f63dcc00700 1 -- 172.21.15.94:0/721481926 >> 172.21.15.94:0/721481926 conn(0x7f63d81033e0 msgr2=0x7f63d8105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.780+0000 7f63dcc00700 1 -- 172.21.15.94:0/721481926 shutdown_connections 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.780+0000 7f63dcc00700 1 -- 172.21.15.94:0/721481926 wait complete. 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.781+0000 7f63dcc00700 1 Processor -- start 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.781+0000 7f63dcc00700 1 -- start start 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.782+0000 7f63dcc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d807db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.782+0000 7f63dcc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f63d810c390 con 0x7f63d8107de0 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.782+0000 7f63d5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.782+0000 7f63d5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50758/0 (socket says 172.21.15.94:50758) 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.782+0000 7f63d5e00700 1 -- 172.21.15.94:0/212793687 learned_addr learned my addr 172.21.15.94:0/212793687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.782+0000 7f63d5e00700 1 -- 172.21.15.94:0/212793687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d807e070 con 0x7f63d8107de0 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63d5e00700 1 --2- 172.21.15.94:0/212793687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d807db30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000c00 tx=0x7f63c8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63cea00700 1 -- 172.21.15.94:0/212793687 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63c8003b20 con 0x7f63d8107de0 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63cea00700 1 -- 172.21.15.94:0/212793687 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63c8003cc0 con 0x7f63d8107de0 2024-06-03T00:55:37.534 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d807c260 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63cea00700 1 -- 172.21.15.94:0/212793687 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63c8024910 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d807c7a0 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63cea00700 1 -- 172.21.15.94:0/212793687 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f63c8022020 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.783+0000 7f63cea00700 1 -- 172.21.15.94:0/212793687 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63c8011e00 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.784+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d8063af0 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.787+0000 7f63cea00700 1 -- 172.21.15.94:0/212793687 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f63c8042dd0 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.867+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f63d807ccd0 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.867+0000 7f63cea00700 1 -- 172.21.15.94:0/212793687 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f63c8031020 con 0x7f63d8107de0 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.868+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 msgr2=0x7f63d807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.868+0000 7f63dcc00700 1 --2- 172.21.15.94:0/212793687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d807db30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000c00 tx=0x7f63c8011c10 comp rx=0 tx=0).stop 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.868+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 shutdown_connections 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.868+0000 7f63dcc00700 1 --2- 172.21.15.94:0/212793687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f63d8107de0 0x7f63d807db30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:37.535 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.868+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 >> 172.21.15.94:0/212793687 conn(0x7f63d81033e0 msgr2=0x7f63d8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:37.536 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.869+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 shutdown_connections 2024-06-03T00:55:37.536 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:36.869+0000 7f63dcc00700 1 -- 172.21.15.94:0/212793687 wait complete. 2024-06-03T00:55:37.536 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-06-03T00:55:38.808 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:38.373+0000 7fc9a95d7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T00:55:38.809 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:38.597+0000 7fc9a95d7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T00:55:38.809 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:38.807+0000 7fc9a95d7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T00:55:39.555 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:39.183+0000 7fc9a95d7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T00:55:39.555 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:39.306+0000 7fc9a95d7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T00:55:39.813 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:39.554+0000 7fc9a95d7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T00:55:40.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:39.812+0000 7fc9a95d7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:55:40.598 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:40 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/354213879' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T00:55:40.988 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:40.626+0000 7fc9a95d7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T00:55:41.000 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:41.001 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T00:55:41.002 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T00:55:41.002 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T00:55:41.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T00:55:41.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T00:55:41.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T00:55:41.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T00:55:41.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T00:55:41.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T00:55:25.857646+0000", 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.136+0000 7f0258c00700 1 Processor -- start 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.136+0000 7f0258c00700 1 -- start start 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.137+0000 7f0258c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.137+0000 7f0258c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f025407c8b0 con 0x7f025407de30 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.137+0000 7f0251e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.137+0000 7f0251e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50762/0 (socket says 172.21.15.94:50762) 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.137+0000 7f0251e00700 1 -- 172.21.15.94:0/3202934743 learned_addr learned my addr 172.21.15.94:0/3202934743 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:41.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.138+0000 7f0251e00700 1 -- 172.21.15.94:0/3202934743 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025407ca30 con 0x7f025407de30 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.138+0000 7f0251e00700 1 --2- 172.21.15.94:0/3202934743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025407c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02440098a0 tx=0x7f0244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49431d5b056d5f0d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.138+0000 7f0250a00700 1 -- 172.21.15.94:0/3202934743 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0244004160 con 0x7f025407de30 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.138+0000 7f0250a00700 1 -- 172.21.15.94:0/3202934743 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0244004300 con 0x7f025407de30 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.138+0000 7f0250a00700 1 -- 172.21.15.94:0/3202934743 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0244011b50 con 0x7f025407de30 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.139+0000 7f0258c00700 1 -- 172.21.15.94:0/3202934743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 msgr2=0x7f025407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.139+0000 7f0258c00700 1 --2- 172.21.15.94:0/3202934743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025407c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f02440098a0 tx=0x7f0244009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.139+0000 7f0258c00700 1 -- 172.21.15.94:0/3202934743 shutdown_connections 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.139+0000 7f0258c00700 1 --2- 172.21.15.94:0/3202934743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025407c2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.139+0000 7f0258c00700 1 -- 172.21.15.94:0/3202934743 >> 172.21.15.94:0/3202934743 conn(0x7f02541033a0 msgr2=0x7f02541057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.139+0000 7f0258c00700 1 -- 172.21.15.94:0/3202934743 shutdown_connections 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.139+0000 7f0258c00700 1 -- 172.21.15.94:0/3202934743 wait complete. 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.140+0000 7f0258c00700 1 Processor -- start 2024-06-03T00:55:41.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.141+0000 7f0258c00700 1 -- start start 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.141+0000 7f0258c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.141+0000 7f0258c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f025410e390 con 0x7f025407de30 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.141+0000 7f0251e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.141+0000 7f0251e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50764/0 (socket says 172.21.15.94:50764) 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.141+0000 7f0251e00700 1 -- 172.21.15.94:0/354213879 learned_addr learned my addr 172.21.15.94:0/354213879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.142+0000 7f0251e00700 1 -- 172.21.15.94:0/354213879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025419e470 con 0x7f025407de30 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.142+0000 7f0251e00700 1 --2- 172.21.15.94:0/354213879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025419df30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0244011b60 tx=0x7f0244011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.142+0000 7f024aa00700 1 -- 172.21.15.94:0/354213879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0244003b20 con 0x7f025407de30 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.142+0000 7f024aa00700 1 -- 172.21.15.94:0/354213879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0244003cc0 con 0x7f025407de30 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.142+0000 7f024aa00700 1 -- 172.21.15.94:0/354213879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0244024840 con 0x7f025407de30 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.142+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025419e700 con 0x7f025407de30 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.142+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02541a13f0 con 0x7f025407de30 2024-06-03T00:55:41.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.143+0000 7f024aa00700 1 -- 172.21.15.94:0/354213879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0244022020 con 0x7f025407de30 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.143+0000 7f024aa00700 1 -- 172.21.15.94:0/354213879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0244011e00 con 0x7f025407de30 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.143+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0254063af0 con 0x7f025407de30 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.148+0000 7f024aa00700 1 -- 172.21.15.94:0/354213879 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0244043dd0 con 0x7f025407de30 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.227+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f02541a15c0 con 0x7f025407de30 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.228+0000 7f024aa00700 1 -- 172.21.15.94:0/354213879 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0244032030 con 0x7f025407de30 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.229+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 msgr2=0x7f025419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.229+0000 7f0258c00700 1 --2- 172.21.15.94:0/354213879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025419df30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0244011b60 tx=0x7f0244011c40 comp rx=0 tx=0).stop 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.229+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 shutdown_connections 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.229+0000 7f0258c00700 1 --2- 172.21.15.94:0/354213879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f025407de30 0x7f025419df30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.229+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 >> 172.21.15.94:0/354213879 conn(0x7f02541033a0 msgr2=0x7f0254103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.229+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 shutdown_connections 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:40.229+0000 7f0258c00700 1 -- 172.21.15.94:0/354213879 wait complete. 2024-06-03T00:55:41.015 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-06-03T00:55:41.309 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:41 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:41.180+0000 7fc9a95d7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T00:55:41.309 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:41 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:41.308+0000 7fc9a95d7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T00:55:41.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:41 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:41.569+0000 7fc9a95d7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T00:55:42.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:42.208+0000 7fc9a95d7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T00:55:42.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:42.330+0000 7fc9a95d7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T00:55:42.952 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:42.647+0000 7fc9a95d7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T00:55:42.953 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:42.827+0000 7fc9a95d7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T00:55:42.953 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:42.951+0000 7fc9a95d7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T00:55:43.283 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:43.107+0000 7fc9a95d7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T00:55:43.579 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:43.481+0000 7fc9a95d7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T00:55:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:43 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2060925354' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T00:55:44.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:44.260+0000 7fc9a95d7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T00:55:44.388 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:44.388 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:44.389 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T00:55:44.390 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T00:55:44.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T00:55:44.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T00:55:44.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T00:55:25.857646+0000", 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.578+0000 7f5017000700 1 Processor -- start 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.578+0000 7f5017000700 1 -- start start 2024-06-03T00:55:44.401 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.579+0000 7f5017000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f5010108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.579+0000 7f5017000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50101087d0 con 0x7f5010107de0 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.579+0000 7f500fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f5010108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.579+0000 7f500fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f5010108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50766/0 (socket says 172.21.15.94:50766) 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.579+0000 7f500fe00700 1 -- 172.21.15.94:0/1552524539 learned_addr learned my addr 172.21.15.94:0/1552524539 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.580+0000 7f500fe00700 1 -- 172.21.15.94:0/1552524539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5010108fe0 con 0x7f5010107de0 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.580+0000 7f500fe00700 1 --2- 172.21.15.94:0/1552524539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f5010108200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50000098a0 tx=0x7f5000009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c764c5943464a68 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.580+0000 7f500ea00700 1 -- 172.21.15.94:0/1552524539 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5000004160 con 0x7f5010107de0 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.580+0000 7f500ea00700 1 -- 172.21.15.94:0/1552524539 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5000004300 con 0x7f5010107de0 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.580+0000 7f500ea00700 1 -- 172.21.15.94:0/1552524539 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5000011b50 con 0x7f5010107de0 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.581+0000 7f5017000700 1 -- 172.21.15.94:0/1552524539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 msgr2=0x7f5010108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:44.402 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.581+0000 7f5017000700 1 --2- 172.21.15.94:0/1552524539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f5010108200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50000098a0 tx=0x7f5000009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.581+0000 7f5017000700 1 -- 172.21.15.94:0/1552524539 shutdown_connections 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.581+0000 7f5017000700 1 --2- 172.21.15.94:0/1552524539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f5010108200 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.581+0000 7f5017000700 1 -- 172.21.15.94:0/1552524539 >> 172.21.15.94:0/1552524539 conn(0x7f50101033e0 msgr2=0x7f5010105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.582+0000 7f5017000700 1 -- 172.21.15.94:0/1552524539 shutdown_connections 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.582+0000 7f5017000700 1 -- 172.21.15.94:0/1552524539 wait complete. 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.582+0000 7f5017000700 1 Processor -- start 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.583+0000 7f5017000700 1 -- start start 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.583+0000 7f5017000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f501007db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.583+0000 7f5017000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f501010c390 con 0x7f5010107de0 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.583+0000 7f500fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f501007db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.583+0000 7f500fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f501007db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50770/0 (socket says 172.21.15.94:50770) 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.583+0000 7f500fe00700 1 -- 172.21.15.94:0/2060925354 learned_addr learned my addr 172.21.15.94:0/2060925354 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:44.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.584+0000 7f500fe00700 1 -- 172.21.15.94:0/2060925354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501007e070 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.584+0000 7f500fe00700 1 --2- 172.21.15.94:0/2060925354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f501007db30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f5000011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.584+0000 7f500cc00700 1 -- 172.21.15.94:0/2060925354 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5000003b20 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.584+0000 7f500cc00700 1 -- 172.21.15.94:0/2060925354 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5000003cc0 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.584+0000 7f500cc00700 1 -- 172.21.15.94:0/2060925354 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5000024910 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.584+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501007c260 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.584+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501007c7a0 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.585+0000 7f500cc00700 1 -- 172.21.15.94:0/2060925354 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5000022020 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.585+0000 7f500cc00700 1 -- 172.21.15.94:0/2060925354 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5000011e00 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.586+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5010063af0 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.588+0000 7f500cc00700 1 -- 172.21.15.94:0/2060925354 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5000042dd0 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.667+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f501007ccd0 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.668+0000 7f500cc00700 1 -- 172.21.15.94:0/2060925354 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f5000031020 con 0x7f5010107de0 2024-06-03T00:55:44.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.669+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 msgr2=0x7f501007db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:44.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.669+0000 7f5017000700 1 --2- 172.21.15.94:0/2060925354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f501007db30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f5000011c10 comp rx=0 tx=0).stop 2024-06-03T00:55:44.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.670+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 shutdown_connections 2024-06-03T00:55:44.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.670+0000 7f5017000700 1 --2- 172.21.15.94:0/2060925354 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5010107de0 0x7f501007db30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:44.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.670+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 >> 172.21.15.94:0/2060925354 conn(0x7f50101033e0 msgr2=0x7f5010103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:44.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.670+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 shutdown_connections 2024-06-03T00:55:44.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:43.670+0000 7f5017000700 1 -- 172.21.15.94:0/2060925354 wait complete. 2024-06-03T00:55:44.405 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (3/15)... 2024-06-03T00:55:44.721 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:44.550+0000 7fc9a95d7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T00:55:45.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: Activating manager daemon x 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: mgrmap e2: x(active, starting, since 0.0118803s) 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: Manager daemon x is now available 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' 2024-06-03T00:55:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:44 smithi094 ceph-mon[32119]: from='mgr.14100 172.21.15.94:0/2040174154' entity='mgr.x' 2024-06-03T00:55:46.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:46 smithi094 ceph-mon[32119]: mgrmap e3: x(active, since 1.02098s) 2024-06-03T00:55:47.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:47 smithi094 ceph-mon[32119]: mgrmap e4: x(active, since 2s) 2024-06-03T00:55:47.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:47 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3152755720' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-03T00:55:48.003 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-03T00:55:48.004 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-03T00:55:48.005 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:48.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-03T00:55:48.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:48.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-06-03T00:55:25.857646+0000", 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.026+0000 7f7573000700 1 Processor -- start 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.026+0000 7f7573000700 1 -- start start 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.026+0000 7f7573000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c1080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.026+0000 7f7573000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1086b0 con 0x7f756c107cc0 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.027+0000 7f7571c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c1080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.027+0000 7f7571c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c1080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41402/0 (socket says 172.21.15.94:41402) 2024-06-03T00:55:48.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.027+0000 7f7571c00700 1 -- 172.21.15.94:0/4131422341 learned_addr learned my addr 172.21.15.94:0/4131422341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.027+0000 7f7571c00700 1 -- 172.21.15.94:0/4131422341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c108e20 con 0x7f756c107cc0 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.028+0000 7f7571c00700 1 --2- 172.21.15.94:0/4131422341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c1080e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f75600098a0 tx=0x7f7560009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acf2d0583ee9a4e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.028+0000 7f756be00700 1 -- 172.21.15.94:0/4131422341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7560004160 con 0x7f756c107cc0 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.028+0000 7f756be00700 1 -- 172.21.15.94:0/4131422341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7560004300 con 0x7f756c107cc0 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.028+0000 7f756be00700 1 -- 172.21.15.94:0/4131422341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7560011b50 con 0x7f756c107cc0 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.029+0000 7f7573000700 1 -- 172.21.15.94:0/4131422341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 msgr2=0x7f756c1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.029+0000 7f7573000700 1 --2- 172.21.15.94:0/4131422341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c1080e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f75600098a0 tx=0x7f7560009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.029+0000 7f7573000700 1 -- 172.21.15.94:0/4131422341 shutdown_connections 2024-06-03T00:55:48.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.029+0000 7f7573000700 1 --2- 172.21.15.94:0/4131422341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c1080e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.029+0000 7f7573000700 1 -- 172.21.15.94:0/4131422341 >> 172.21.15.94:0/4131422341 conn(0x7f756c1032a0 msgr2=0x7f756c1056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.029+0000 7f7573000700 1 -- 172.21.15.94:0/4131422341 shutdown_connections 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.029+0000 7f7573000700 1 -- 172.21.15.94:0/4131422341 wait complete. 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.030+0000 7f7573000700 1 Processor -- start 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.030+0000 7f7573000700 1 -- start start 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.030+0000 7f7573000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.030+0000 7f7573000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f756c10c1d0 con 0x7f756c107cc0 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.031+0000 7f7571c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.031+0000 7f7571c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41404/0 (socket says 172.21.15.94:41404) 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.031+0000 7f7571c00700 1 -- 172.21.15.94:0/3152755720 learned_addr learned my addr 172.21.15.94:0/3152755720 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.031+0000 7f7571c00700 1 -- 172.21.15.94:0/3152755720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c19e3b0 con 0x7f756c107cc0 2024-06-03T00:55:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.031+0000 7f7571c00700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c19de70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fb0 tx=0x7f7560011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.031+0000 7f756a000700 1 -- 172.21.15.94:0/3152755720 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75600038b0 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.031+0000 7f756a000700 1 -- 172.21.15.94:0/3152755720 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7560011e50 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.032+0000 7f756a000700 1 -- 172.21.15.94:0/3152755720 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7560019550 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.032+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c19e640 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.032+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c1a1330 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.033+0000 7f756a000700 1 -- 172.21.15.94:0/3152755720 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f756001a430 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.033+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f756c063af0 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.033+0000 7f756a000700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f7554038fb0 0x7f755403b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.033+0000 7f756a000700 1 -- 172.21.15.94:0/3152755720 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f756004d270 con 0x7f756c107cc0 2024-06-03T00:55:48.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.033+0000 7f7571200700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f7554038fb0 0x7f755403b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.034+0000 7f7571200700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f7554038fb0 0x7f755403b470 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7558006fd0 tx=0x7f7558006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.040+0000 7f756a000700 1 -- 172.21.15.94:0/3152755720 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7560020240 con 0x7f756c107cc0 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.347+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f756c1a1500 con 0x7f756c107cc0 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.348+0000 7f756a000700 1 -- 172.21.15.94:0/3152755720 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f756c1a1500 con 0x7f756c107cc0 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f7554038fb0 msgr2=0x7f755403b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f7554038fb0 0x7f755403b470 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7558006fd0 tx=0x7f7558006e40 comp rx=0 tx=0).stop 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 msgr2=0x7f756c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c19de70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7560009fb0 tx=0x7f7560011bf0 comp rx=0 tx=0).stop 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 shutdown_connections 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f7554038fb0 0x7f755403b470 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:48.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 --2- 172.21.15.94:0/3152755720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f756c107cc0 0x7f756c19de70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:48.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 >> 172.21.15.94:0/3152755720 conn(0x7f756c1032a0 msgr2=0x7f756c103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:48.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 shutdown_connections 2024-06-03T00:55:48.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:47.351+0000 7f7573000700 1 -- 172.21.15.94:0/3152755720 wait complete. 2024-06-03T00:55:48.014 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-06-03T00:55:49.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:48 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2329418142' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-03T00:55:49.559 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-03T00:55:49.560 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.641+0000 7fbce5000700 1 Processor -- start 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.641+0000 7fbce5000700 1 -- start start 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.642+0000 7fbce5000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce0108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.642+0000 7fbce5000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01087b0 con 0x7fbce0107e50 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.642+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce0108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.643+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce0108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41420/0 (socket says 172.21.15.94:41420) 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.643+0000 7fbcdde00700 1 -- 172.21.15.94:0/866970799 learned_addr learned my addr 172.21.15.94:0/866970799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:49.561 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.643+0000 7fbcdde00700 1 -- 172.21.15.94:0/866970799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce0108fb0 con 0x7fbce0107e50 2024-06-03T00:55:49.562 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.643+0000 7fbcdde00700 1 --2- 172.21.15.94:0/866970799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce0108270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098a0 tx=0x7fbcd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a07981d4915edc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:49.562 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.644+0000 7fbcdca00700 1 -- 172.21.15.94:0/866970799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0004160 con 0x7fbce0107e50 2024-06-03T00:55:49.562 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.644+0000 7fbcdca00700 1 -- 172.21.15.94:0/866970799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcd0004300 con 0x7fbce0107e50 2024-06-03T00:55:49.562 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.644+0000 7fbcdca00700 1 -- 172.21.15.94:0/866970799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0011b50 con 0x7fbce0107e50 2024-06-03T00:55:49.562 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.644+0000 7fbce5000700 1 -- 172.21.15.94:0/866970799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 msgr2=0x7fbce0108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:49.562 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.644+0000 7fbce5000700 1 --2- 172.21.15.94:0/866970799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce0108270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098a0 tx=0x7fbcd0009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:49.562 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.645+0000 7fbce5000700 1 -- 172.21.15.94:0/866970799 shutdown_connections 2024-06-03T00:55:49.563 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.645+0000 7fbce5000700 1 --2- 172.21.15.94:0/866970799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce0108270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:49.563 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.645+0000 7fbce5000700 1 -- 172.21.15.94:0/866970799 >> 172.21.15.94:0/866970799 conn(0x7fbce0103470 msgr2=0x7fbce0105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:49.563 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.645+0000 7fbce5000700 1 -- 172.21.15.94:0/866970799 shutdown_connections 2024-06-03T00:55:49.563 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.645+0000 7fbce5000700 1 -- 172.21.15.94:0/866970799 wait complete. 2024-06-03T00:55:49.563 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.646+0000 7fbce5000700 1 Processor -- start 2024-06-03T00:55:49.563 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.646+0000 7fbce5000700 1 -- start start 2024-06-03T00:55:49.563 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.646+0000 7fbce5000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.646+0000 7fbce5000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010c270 con 0x7fbce0107e50 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcdde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce019df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41430/0 (socket says 172.21.15.94:41430) 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcdde00700 1 -- 172.21.15.94:0/2329418142 learned_addr learned my addr 172.21.15.94:0/2329418142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcdde00700 1 -- 172.21.15.94:0/2329418142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce019e4a0 con 0x7fbce0107e50 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcdde00700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce019df60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0011b60 tx=0x7fbcd0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcd6a00700 1 -- 172.21.15.94:0/2329418142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0003b20 con 0x7fbce0107e50 2024-06-03T00:55:49.564 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcd6a00700 1 -- 172.21.15.94:0/2329418142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcd0003cc0 con 0x7fbce0107e50 2024-06-03T00:55:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbcd6a00700 1 -- 172.21.15.94:0/2329418142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd0024840 con 0x7fbce0107e50 2024-06-03T00:55:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.647+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce019e730 con 0x7fbce0107e50 2024-06-03T00:55:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.648+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01a1420 con 0x7fbce0107e50 2024-06-03T00:55:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.649+0000 7fbcd6a00700 1 -- 172.21.15.94:0/2329418142 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbcd0022020 con 0x7fbce0107e50 2024-06-03T00:55:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.649+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce010c3f0 con 0x7fbce0107e50 2024-06-03T00:55:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.649+0000 7fbcd6a00700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fbcc4039050 0x7fbcc403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:49.565 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.649+0000 7fbcd6a00700 1 -- 172.21.15.94:0/2329418142 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbcd0011e00 con 0x7fbce0107e50 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.649+0000 7fbcdd400700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fbcc4039050 0x7fbcc403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.650+0000 7fbcdd400700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fbcc4039050 0x7fbcc403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.655+0000 7fbcd6a00700 1 -- 172.21.15.94:0/2329418142 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbcd004cae0 con 0x7fbce0107e50 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.878+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbce0063af0 con 0x7fbce0107e50 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.880+0000 7fbcd6a00700 1 -- 172.21.15.94:0/2329418142 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fbcd004ce20 con 0x7fbce0107e50 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fbcc4039050 msgr2=0x7fbcc403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fbcc4039050 0x7fbcc403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).stop 2024-06-03T00:55:49.566 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 msgr2=0x7fbce019df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce019df60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0011b60 tx=0x7fbcd0011c40 comp rx=0 tx=0).stop 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 shutdown_connections 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fbcc4039050 0x7fbcc403b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 --2- 172.21.15.94:0/2329418142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbce0107e50 0x7fbce019df60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 >> 172.21.15.94:0/2329418142 conn(0x7fbce0103470 msgr2=0x7fbce0103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.883+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 shutdown_connections 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:48.884+0000 7fbce5000700 1 -- 172.21.15.94:0/2329418142 wait complete. 2024-06-03T00:55:49.567 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-06-03T00:55:50.754 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:50 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/823663381' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-03T00:55:50.754 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: ignoring --setuser ceph since I am not root 2024-06-03T00:55:50.754 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: ignoring --setgroup ceph since I am not root 2024-06-03T00:55:51.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:50.753+0000 7fe69fb99200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T00:55:51.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:50.996+0000 7fe69fb99200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:55:51.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.129+0000 7f80c5800700 1 Processor -- start 2024-06-03T00:55:51.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.129+0000 7f80c5800700 1 -- start start 2024-06-03T00:55:51.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.129+0000 7f80c5800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c0081f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:51.239 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.129+0000 7f80c5800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0082530 con 0x7f80c0081b40 2024-06-03T00:55:51.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.130+0000 7f80bfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c0081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:51.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.130+0000 7f80bfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c0081f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41438/0 (socket says 172.21.15.94:41438) 2024-06-03T00:55:51.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.130+0000 7f80bfe00700 1 -- 172.21.15.94:0/2001175134 learned_addr learned my addr 172.21.15.94:0/2001175134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:51.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.130+0000 7f80bfe00700 1 -- 172.21.15.94:0/2001175134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c010f520 con 0x7f80c0081b40 2024-06-03T00:55:51.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.131+0000 7f80bfe00700 1 --2- 172.21.15.94:0/2001175134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c0081f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f80b00098a0 tx=0x7f80b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8288838796c6ad1b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.131+0000 7f80bea00700 1 -- 172.21.15.94:0/2001175134 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b0004160 con 0x7f80c0081b40 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.131+0000 7f80bea00700 1 -- 172.21.15.94:0/2001175134 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80b0004300 con 0x7f80c0081b40 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.131+0000 7f80bea00700 1 -- 172.21.15.94:0/2001175134 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b0011b50 con 0x7f80c0081b40 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.132+0000 7f80c5800700 1 -- 172.21.15.94:0/2001175134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 msgr2=0x7f80c0081f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.132+0000 7f80c5800700 1 --2- 172.21.15.94:0/2001175134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c0081f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f80b00098a0 tx=0x7f80b0009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.132+0000 7f80c5800700 1 -- 172.21.15.94:0/2001175134 shutdown_connections 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.132+0000 7f80c5800700 1 --2- 172.21.15.94:0/2001175134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c0081f60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.132+0000 7f80c5800700 1 -- 172.21.15.94:0/2001175134 >> 172.21.15.94:0/2001175134 conn(0x7f80c007d1b0 msgr2=0x7f80c007f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.132+0000 7f80c5800700 1 -- 172.21.15.94:0/2001175134 shutdown_connections 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.132+0000 7f80c5800700 1 -- 172.21.15.94:0/2001175134 wait complete. 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.133+0000 7f80c5800700 1 Processor -- start 2024-06-03T00:55:51.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.133+0000 7f80c5800700 1 -- start start 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.134+0000 7f80c5800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c01a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.134+0000 7f80c5800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0112750 con 0x7f80c0081b40 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.134+0000 7f80bfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c01a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.134+0000 7f80bfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c01a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41446/0 (socket says 172.21.15.94:41446) 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.134+0000 7f80bfe00700 1 -- 172.21.15.94:0/823663381 learned_addr learned my addr 172.21.15.94:0/823663381 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.134+0000 7f80bfe00700 1 -- 172.21.15.94:0/823663381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c01a26e0 con 0x7f80c0081b40 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.135+0000 7f80bfe00700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c01a21a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f80b0011670 tx=0x7f80b0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.135+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b00039c0 con 0x7f80c0081b40 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.135+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80b0011e50 con 0x7f80c0081b40 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.135+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c01a2970 con 0x7f80c0081b40 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.135+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b001a5c0 con 0x7f80c0081b40 2024-06-03T00:55:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.135+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c01a5660 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.136+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f80b001a870 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.136+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c0063af0 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.137+0000 7f80bcc00700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f80a4039000 0x7f80a403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.137+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80b004d4e0 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.137+0000 7f80bf400700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f80a4039000 0x7f80a403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.138+0000 7f80bf400700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f80a4039000 0x7f80a403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.142+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f80b004cf00 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.421+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f80c01a5830 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.475+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f80c01a5830 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.478+0000 7f80bcc00700 1 -- 172.21.15.94:0/823663381 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f80b002d3f0 con 0x7f80c0081b40 2024-06-03T00:55:51.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f80a4039000 msgr2=0x7f80a403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f80a4039000 0x7f80a403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f80ac006fd0 tx=0x7f80ac006e40 comp rx=0 tx=0).stop 2024-06-03T00:55:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 msgr2=0x7f80c01a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c01a21a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f80b0011670 tx=0x7f80b0011b60 comp rx=0 tx=0).stop 2024-06-03T00:55:51.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 shutdown_connections 2024-06-03T00:55:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f80a4039000 0x7f80a403b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 --2- 172.21.15.94:0/823663381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80c0081b40 0x7f80c01a21a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.479+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 >> 172.21.15.94:0/823663381 conn(0x7f80c007d1b0 msgr2=0x7f80c007dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.480+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 shutdown_connections 2024-06-03T00:55:51.249 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:50.481+0000 7f80c5800700 1 -- 172.21.15.94:0/823663381 wait complete. 2024-06-03T00:55:51.395 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:51.128+0000 7fe69fb99200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T00:55:51.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:51 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/823663381' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-03T00:55:51.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:51 smithi094 ceph-mon[32119]: mgrmap e5: x(active, since 5s) 2024-06-03T00:55:51.655 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:51.540+0000 7fe69fb99200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T00:55:52.027 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:51.654+0000 7fe69fb99200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T00:55:52.775 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:52 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2446007948' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T00:55:52.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:55:52.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-03T00:55:52.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T00:55:52.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-03T00:55:52.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T00:55:52.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:55:52.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.818+0000 7f344f200700 1 Processor -- start 2024-06-03T00:55:52.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.818+0000 7f344f200700 1 -- start start 2024-06-03T00:55:52.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.818+0000 7f344f200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448106f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:52.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.818+0000 7f344f200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34481074d0 con 0x7f3448106ae0 2024-06-03T00:55:52.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.819+0000 7f344de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:52.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.819+0000 7f344de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41478/0 (socket says 172.21.15.94:41478) 2024-06-03T00:55:52.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.819+0000 7f344de00700 1 -- 172.21.15.94:0/1262156647 learned_addr learned my addr 172.21.15.94:0/1262156647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:52.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.819+0000 7f344de00700 1 -- 172.21.15.94:0/1262156647 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3448107ce0 con 0x7f3448106ae0 2024-06-03T00:55:52.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.819+0000 7f344de00700 1 --2- 172.21.15.94:0/1262156647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448106f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f343c0098a0 tx=0x7f343c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bd0c3e3bebe9ae server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:52.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.820+0000 7f344ca00700 1 -- 172.21.15.94:0/1262156647 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f343c004160 con 0x7f3448106ae0 2024-06-03T00:55:52.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.820+0000 7f344ca00700 1 -- 172.21.15.94:0/1262156647 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f343c004300 con 0x7f3448106ae0 2024-06-03T00:55:52.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.820+0000 7f344ca00700 1 -- 172.21.15.94:0/1262156647 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f343c011b50 con 0x7f3448106ae0 2024-06-03T00:55:52.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.820+0000 7f344f200700 1 -- 172.21.15.94:0/1262156647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 msgr2=0x7f3448106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:52.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.820+0000 7f344f200700 1 --2- 172.21.15.94:0/1262156647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448106f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f343c0098a0 tx=0x7f343c009bb0 comp rx=0 tx=0).stop 2024-06-03T00:55:52.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.821+0000 7f344f200700 1 -- 172.21.15.94:0/1262156647 shutdown_connections 2024-06-03T00:55:52.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.821+0000 7f344f200700 1 --2- 172.21.15.94:0/1262156647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448106f00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:52.798 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.821+0000 7f344f200700 1 -- 172.21.15.94:0/1262156647 >> 172.21.15.94:0/1262156647 conn(0x7f34481020a0 msgr2=0x7f34481044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:52.798 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.821+0000 7f344f200700 1 -- 172.21.15.94:0/1262156647 shutdown_connections 2024-06-03T00:55:52.798 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.821+0000 7f344f200700 1 -- 172.21.15.94:0/1262156647 wait complete. 2024-06-03T00:55:52.798 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.822+0000 7f344f200700 1 Processor -- start 2024-06-03T00:55:52.798 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.822+0000 7f344f200700 1 -- start start 2024-06-03T00:55:52.798 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.822+0000 7f344f200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448191c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:52.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.822+0000 7f344f200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f344810b090 con 0x7f3448106ae0 2024-06-03T00:55:52.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.822+0000 7f344de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448191c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:55:52.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.822+0000 7f344de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448191c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41480/0 (socket says 172.21.15.94:41480) 2024-06-03T00:55:52.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.823+0000 7f344de00700 1 -- 172.21.15.94:0/2446007948 learned_addr learned my addr 172.21.15.94:0/2446007948 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:55:52.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.823+0000 7f344de00700 1 -- 172.21.15.94:0/2446007948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34481921d0 con 0x7f3448106ae0 2024-06-03T00:55:52.799 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.823+0000 7f344de00700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448191c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f343c000c00 tx=0x7f343c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:55:52.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.823+0000 7f3442a00700 1 -- 172.21.15.94:0/2446007948 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f343c003b20 con 0x7f3448106ae0 2024-06-03T00:55:52.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.823+0000 7f3442a00700 1 -- 172.21.15.94:0/2446007948 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f343c003cc0 con 0x7f3448106ae0 2024-06-03T00:55:52.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.823+0000 7f344f200700 1 -- 172.21.15.94:0/2446007948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34481903c0 con 0x7f3448106ae0 2024-06-03T00:55:52.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.823+0000 7f3442a00700 1 -- 172.21.15.94:0/2446007948 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f343c024910 con 0x7f3448106ae0 2024-06-03T00:55:52.800 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.824+0000 7f344f200700 1 -- 172.21.15.94:0/2446007948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3448190900 con 0x7f3448106ae0 2024-06-03T00:55:52.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.825+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3448063af0 con 0x7f3448106ae0 2024-06-03T00:55:52.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.826+0000 7f3442a00700 1 -- 172.21.15.94:0/2446007948 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f343c022020 con 0x7f3448106ae0 2024-06-03T00:55:52.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.826+0000 7f3442a00700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 0x7f343003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:55:52.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.826+0000 7f3442a00700 1 -- 172.21.15.94:0/2446007948 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f343c011e00 con 0x7f3448106ae0 2024-06-03T00:55:52.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.826+0000 7f344d400700 1 -- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 msgr2=0x7f343003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2467202876 2024-06-03T00:55:52.801 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.826+0000 7f344d400700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 0x7f343003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T00:55:52.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:51.831+0000 7f3442a00700 1 -- 172.21.15.94:0/2446007948 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f343c019df0 con 0x7f3448106ae0 2024-06-03T00:55:52.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.027+0000 7f344d400700 1 -- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 msgr2=0x7f343003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2467202876 2024-06-03T00:55:52.802 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.027+0000 7f344d400700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 0x7f343003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T00:55:52.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.109+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f34481a3450 con 0x7f3448106ae0 2024-06-03T00:55:52.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.110+0000 7f3442a00700 1 -- 172.21.15.94:0/2446007948 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f343c01ad80 con 0x7f3448106ae0 2024-06-03T00:55:52.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 msgr2=0x7f343003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T00:55:52.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 0x7f343003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:52.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 msgr2=0x7f3448191c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:55:52.803 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448191c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f343c000c00 tx=0x7f343c011c10 comp rx=0 tx=0).stop 2024-06-03T00:55:52.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 shutdown_connections 2024-06-03T00:55:52.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7f34300390a0 0x7f343003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:52.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 --2- 172.21.15.94:0/2446007948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3448106ae0 0x7f3448191c90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:55:52.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 >> 172.21.15.94:0/2446007948 conn(0x7f34481020a0 msgr2=0x7f3448102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:55:52.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 shutdown_connections 2024-06-03T00:55:52.804 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:52.113+0000 7f3437e00700 1 -- 172.21.15.94:0/2446007948 wait complete. 2024-06-03T00:55:52.805 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-06-03T00:55:52.805 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-06-03T00:55:53.361 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:53.212+0000 7fe69fb99200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T00:55:53.658 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:53.443+0000 7fe69fb99200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T00:55:54.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:53.657+0000 7fe69fb99200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T00:55:54.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:54.035+0000 7fe69fb99200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T00:55:54.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:54.157+0000 7fe69fb99200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T00:55:54.794 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:54.408+0000 7fe69fb99200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T00:55:54.794 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:54.656+0000 7fe69fb99200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:55:55.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:55.431+0000 7fe69fb99200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T00:55:56.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:55.975+0000 7fe69fb99200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T00:55:56.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:56.104+0000 7fe69fb99200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T00:55:56.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:56.367+0000 7fe69fb99200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T00:55:57.152 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:57.029+0000 7fe69fb99200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T00:55:57.152 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:57.151+0000 7fe69fb99200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T00:55:57.779 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:57.471+0000 7fe69fb99200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T00:55:57.779 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:57.651+0000 7fe69fb99200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T00:55:57.779 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:57.778+0000 7fe69fb99200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T00:55:58.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:57.932+0000 7fe69fb99200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T00:55:58.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:58 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:58.325+0000 7fe69fb99200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T00:55:59.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:59.070+0000 7fe69fb99200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T00:55:59.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:55:59.355+0000 7fe69fb99200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T00:55:59.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: Active manager daemon x restarted 2024-06-03T00:55:59.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: Activating manager daemon x 2024-06-03T00:55:59.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: osdmap e2: 0 total, 0 up, 0 in 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: mgrmap e6: x(active, starting, since 0.00962357s) 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: Manager daemon x is now available 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:55:59.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:55:59 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:00.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: Found migration_current of "None". Setting to last migration. 2024-06-03T00:56:00.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T00:56:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T00:56:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: mgrmap e7: x(active, since 1.01819s) 2024-06-03T00:56:00.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:00 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:01.092 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:56:01.092 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-03T00:56:01.092 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T00:56:01.092 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:56:01.092 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.382+0000 7fc467200700 1 Processor -- start 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.383+0000 7fc467200700 1 -- start start 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.383+0000 7fc467200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.383+0000 7fc467200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc46007c870 con 0x7fc46007de40 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.383+0000 7fc465e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.384+0000 7fc465e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41492/0 (socket says 172.21.15.94:41492) 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.384+0000 7fc465e00700 1 -- 172.21.15.94:0/37952314 learned_addr learned my addr 172.21.15.94:0/37952314 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.384+0000 7fc465e00700 1 -- 172.21.15.94:0/37952314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46007c9f0 con 0x7fc46007de40 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.384+0000 7fc465e00700 1 --2- 172.21.15.94:0/37952314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46007c2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4540098a0 tx=0x7fc454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60562beff165ecca server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.385+0000 7fc464a00700 1 -- 172.21.15.94:0/37952314 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc454004160 con 0x7fc46007de40 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.385+0000 7fc464a00700 1 -- 172.21.15.94:0/37952314 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc454004300 con 0x7fc46007de40 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.385+0000 7fc464a00700 1 -- 172.21.15.94:0/37952314 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc454011b50 con 0x7fc46007de40 2024-06-03T00:56:01.093 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.385+0000 7fc467200700 1 -- 172.21.15.94:0/37952314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 msgr2=0x7fc46007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.385+0000 7fc467200700 1 --2- 172.21.15.94:0/37952314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46007c2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4540098a0 tx=0x7fc454009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.386+0000 7fc467200700 1 -- 172.21.15.94:0/37952314 shutdown_connections 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.386+0000 7fc467200700 1 --2- 172.21.15.94:0/37952314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46007c2a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.386+0000 7fc467200700 1 -- 172.21.15.94:0/37952314 >> 172.21.15.94:0/37952314 conn(0x7fc4601031d0 msgr2=0x7fc4601055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.386+0000 7fc467200700 1 -- 172.21.15.94:0/37952314 shutdown_connections 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.386+0000 7fc467200700 1 -- 172.21.15.94:0/37952314 wait complete. 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.387+0000 7fc467200700 1 Processor -- start 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.387+0000 7fc467200700 1 -- start start 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.387+0000 7fc467200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.387+0000 7fc467200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc46010e220 con 0x7fc46007de40 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.388+0000 7fc465e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.388+0000 7fc465e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46019de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41496/0 (socket says 172.21.15.94:41496) 2024-06-03T00:56:01.094 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.388+0000 7fc465e00700 1 -- 172.21.15.94:0/3586277204 learned_addr learned my addr 172.21.15.94:0/3586277204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.388+0000 7fc465e00700 1 -- 172.21.15.94:0/3586277204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46019e340 con 0x7fc46007de40 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.388+0000 7fc465e00700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46019de00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc454011b60 tx=0x7fc454011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.389+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc454003970 con 0x7fc46007de40 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.389+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc454003b10 con 0x7fc46007de40 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.389+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc454024610 con 0x7fc46007de40 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.389+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46019e5d0 con 0x7fc46007de40 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.389+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a12c0 con 0x7fc46007de40 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.390+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc4540248c0 con 0x7fc46007de40 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.390+0000 7fc45aa00700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.390+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 --> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc44803bc70 con 0x7fc4480390a0 2024-06-03T00:56:01.095 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.390+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc454011e00 con 0x7fc46007de40 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.390+0000 7fc465400700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 msgr2=0x7fc44803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2467202876 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.390+0000 7fc465400700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.591+0000 7fc465400700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 msgr2=0x7fc44803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2467202876 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.591+0000 7fc465400700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.992+0000 7fc465400700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 msgr2=0x7fc44803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2467202876 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:53.992+0000 7fc465400700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:54.793+0000 7fc465400700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 msgr2=0x7fc44803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2467202876 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:54.793+0000 7fc465400700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:56.395+0000 7fc465400700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 msgr2=0x7fc44803b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2467202876 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:56.395+0000 7fc465400700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:59.365+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fc454024bb0 con 0x7fc46007de40 2024-06-03T00:56:01.096 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:59.365+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 msgr2=0x7fc44803b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:55:59.365+0000 7fc45aa00700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.370+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fc45404f0e0 con 0x7fc46007de40 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.371+0000 7fc45aa00700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fc44803c9d0 0x7fc44803edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.371+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc454016e70 con 0x7fc44803c9d0 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.372+0000 7fc465400700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fc44803c9d0 0x7fc44803edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.374+0000 7fc465400700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fc44803c9d0 0x7fc44803edc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc450003a10 tx=0x7fc4500092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.376+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fc454016e70 con 0x7fc44803c9d0 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.387+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc460063af0 con 0x7fc44803c9d0 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.387+0000 7fc45aa00700 1 -- 172.21.15.94:0/3586277204 <== mgr.14120 v2:172.21.15.94:6800/127652834 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc460063af0 con 0x7fc44803c9d0 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fc44803c9d0 msgr2=0x7fc44803edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fc44803c9d0 0x7fc44803edc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc450003a10 tx=0x7fc4500092b0 comp rx=0 tx=0).stop 2024-06-03T00:56:01.097 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 msgr2=0x7fc46019de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46019de00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc454011b60 tx=0x7fc454011c40 comp rx=0 tx=0).stop 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 shutdown_connections 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fc44803c9d0 0x7fc44803edc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46007de40 0x7fc46019de00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 --2- 172.21.15.94:0/3586277204 >> [v2:172.21.15.94:6800/2467202876,v1:172.21.15.94:6801/2467202876] conn(0x7fc4480390a0 0x7fc44803b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 >> 172.21.15.94:0/3586277204 conn(0x7fc4601031d0 msgr2=0x7fc460103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 shutdown_connections 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:00.388+0000 7fc467200700 1 -- 172.21.15.94:0/3586277204 wait complete. 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-06-03T00:56:01.098 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-06-03T00:56:01.644 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:01 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:00] ENGINE Bus STARTING 2024-06-03T00:56:01.644 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:01 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:00] ENGINE Serving on http://172.21.15.94:8765 2024-06-03T00:56:01.644 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:01 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:00] ENGINE Serving on https://172.21.15.94:7150 2024-06-03T00:56:01.644 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:01 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:00] ENGINE Bus STARTED 2024-06-03T00:56:01.644 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:01 smithi094 ceph-mon[32119]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T00:56:01.644 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:01 smithi094 ceph-mon[32119]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T00:56:02.729 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:02 smithi094 ceph-mon[32119]: mgrmap e8: x(active, since 2s) 2024-06-03T00:56:02.729 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:02 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:02.729 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:02 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:02.741 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.807+0000 7f6a8ee00700 1 Processor -- start 2024-06-03T00:56:02.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.807+0000 7f6a8ee00700 1 -- start start 2024-06-03T00:56:02.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.807+0000 7f6a8ee00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8810a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:02.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.808+0000 7f6a8ee00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810ab00 con 0x7f6a8810a110 2024-06-03T00:56:02.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.808+0000 7f6a87e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8810a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.808+0000 7f6a87e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8810a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56870/0 (socket says 172.21.15.94:56870) 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.808+0000 7f6a87e00700 1 -- 172.21.15.94:0/3015618612 learned_addr learned my addr 172.21.15.94:0/3015618612 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.808+0000 7f6a87e00700 1 -- 172.21.15.94:0/3015618612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8810b200 con 0x7f6a8810a110 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.809+0000 7f6a87e00700 1 --2- 172.21.15.94:0/3015618612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8810a530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a780098a0 tx=0x7f6a78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca01db1d57ac8d14 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.809+0000 7f6a87400700 1 -- 172.21.15.94:0/3015618612 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a78004160 con 0x7f6a8810a110 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.809+0000 7f6a87400700 1 -- 172.21.15.94:0/3015618612 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a78004300 con 0x7f6a8810a110 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.809+0000 7f6a87400700 1 -- 172.21.15.94:0/3015618612 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a78011b50 con 0x7f6a8810a110 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.810+0000 7f6a8ee00700 1 -- 172.21.15.94:0/3015618612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 msgr2=0x7f6a8810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.810+0000 7f6a8ee00700 1 --2- 172.21.15.94:0/3015618612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8810a530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a780098a0 tx=0x7f6a78009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.810+0000 7f6a8ee00700 1 -- 172.21.15.94:0/3015618612 shutdown_connections 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.810+0000 7f6a8ee00700 1 --2- 172.21.15.94:0/3015618612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8810a530 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.810+0000 7f6a8ee00700 1 -- 172.21.15.94:0/3015618612 >> 172.21.15.94:0/3015618612 conn(0x7f6a8807d420 msgr2=0x7f6a8807d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:02.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.810+0000 7f6a8ee00700 1 -- 172.21.15.94:0/3015618612 shutdown_connections 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.810+0000 7f6a8ee00700 1 -- 172.21.15.94:0/3015618612 wait complete. 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.811+0000 7f6a8ee00700 1 Processor -- start 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.812+0000 7f6a8ee00700 1 -- start start 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.812+0000 7f6a8ee00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.812+0000 7f6a8ee00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8810e5d0 con 0x7f6a8810a110 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.812+0000 7f6a87e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.812+0000 7f6a87e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56876/0 (socket says 172.21.15.94:56876) 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.812+0000 7f6a87e00700 1 -- 172.21.15.94:0/4064663240 learned_addr learned my addr 172.21.15.94:0/4064663240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.813+0000 7f6a87e00700 1 -- 172.21.15.94:0/4064663240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8819e490 con 0x7f6a8810a110 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.813+0000 7f6a87e00700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8819df50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011670 tx=0x7f6a78011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.813+0000 7f6a85600700 1 -- 172.21.15.94:0/4064663240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a780039c0 con 0x7f6a8810a110 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.813+0000 7f6a85600700 1 -- 172.21.15.94:0/4064663240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a78011e50 con 0x7f6a8810a110 2024-06-03T00:56:02.748 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.813+0000 7f6a85600700 1 -- 172.21.15.94:0/4064663240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a7801a5c0 con 0x7f6a8810a110 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.813+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8819e720 con 0x7f6a8810a110 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.813+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a881a1410 con 0x7f6a8810a110 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.814+0000 7f6a85600700 1 -- 172.21.15.94:0/4064663240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6a78024430 con 0x7f6a8810a110 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.814+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a88063af0 con 0x7f6a8810a110 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.815+0000 7f6a85600700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f6a6c039000 0x7f6a6c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.815+0000 7f6a85600700 1 -- 172.21.15.94:0/4064663240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a7804d560 con 0x7f6a8810a110 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.816+0000 7f6a7fe00700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f6a6c039000 0x7f6a6c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.816+0000 7f6a7fe00700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f6a6c039000 0x7f6a6c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a70006fd0 tx=0x7f6a70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:01.821+0000 7f6a85600700 1 -- 172.21.15.94:0/4064663240 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a7804ceb0 con 0x7f6a8810a110 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.061+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6a88108460 con 0x7f6a6c039000 2024-06-03T00:56:02.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.071+0000 7f6a85600700 1 -- 172.21.15.94:0/4064663240 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a88108460 con 0x7f6a6c039000 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f6a6c039000 msgr2=0x7f6a6c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f6a6c039000 0x7f6a6c03b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a70006fd0 tx=0x7f6a70006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 msgr2=0x7f6a8819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8819df50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011670 tx=0x7f6a78011b60 comp rx=0 tx=0).stop 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 shutdown_connections 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f6a6c039000 0x7f6a6c03b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 --2- 172.21.15.94:0/4064663240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a8810a110 0x7f6a8819df50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 >> 172.21.15.94:0/4064663240 conn(0x7f6a8807d420 msgr2=0x7f6a88107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 shutdown_connections 2024-06-03T00:56:02.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:02.074+0000 7f6a8ee00700 1 -- 172.21.15.94:0/4064663240 wait complete. 2024-06-03T00:56:03.660 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:03 smithi094 ceph-mon[32119]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:04.319 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-03T00:56:04.319 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.351+0000 7f05a1400700 1 Processor -- start 2024-06-03T00:56:04.319 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.351+0000 7f05a1400700 1 -- start start 2024-06-03T00:56:04.320 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.351+0000 7f05a1400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:04.320 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.351+0000 7f05a1400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f059c07c870 con 0x7f059c07de40 2024-06-03T00:56:04.320 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.352+0000 7f059be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:04.320 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.352+0000 7f059be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56886/0 (socket says 172.21.15.94:56886) 2024-06-03T00:56:04.320 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.352+0000 7f059be00700 1 -- 172.21.15.94:0/1719332143 learned_addr learned my addr 172.21.15.94:0/1719332143 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:04.320 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.352+0000 7f059be00700 1 -- 172.21.15.94:0/1719332143 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c07c9f0 con 0x7f059c07de40 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.353+0000 7f059be00700 1 --2- 172.21.15.94:0/1719332143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c07c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f058c0098a0 tx=0x7f058c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0604fd584b4ba72 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.353+0000 7f059aa00700 1 -- 172.21.15.94:0/1719332143 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f058c004160 con 0x7f059c07de40 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.353+0000 7f059aa00700 1 -- 172.21.15.94:0/1719332143 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f058c004300 con 0x7f059c07de40 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.353+0000 7f059aa00700 1 -- 172.21.15.94:0/1719332143 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f058c011b50 con 0x7f059c07de40 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.354+0000 7f05a1400700 1 -- 172.21.15.94:0/1719332143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 msgr2=0x7f059c07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.354+0000 7f05a1400700 1 --2- 172.21.15.94:0/1719332143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c07c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f058c0098a0 tx=0x7f058c009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.354+0000 7f05a1400700 1 -- 172.21.15.94:0/1719332143 shutdown_connections 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.354+0000 7f05a1400700 1 --2- 172.21.15.94:0/1719332143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c07c2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.354+0000 7f05a1400700 1 -- 172.21.15.94:0/1719332143 >> 172.21.15.94:0/1719332143 conn(0x7f059c1031d0 msgr2=0x7f059c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.354+0000 7f05a1400700 1 -- 172.21.15.94:0/1719332143 shutdown_connections 2024-06-03T00:56:04.325 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.354+0000 7f05a1400700 1 -- 172.21.15.94:0/1719332143 wait complete. 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.355+0000 7f05a1400700 1 Processor -- start 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.356+0000 7f05a1400700 1 -- start start 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.356+0000 7f05a1400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.356+0000 7f05a1400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f059c10e220 con 0x7f059c07de40 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.356+0000 7f059be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.356+0000 7f059be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c19de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56894/0 (socket says 172.21.15.94:56894) 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.356+0000 7f059be00700 1 -- 172.21.15.94:0/629894390 learned_addr learned my addr 172.21.15.94:0/629894390 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.357+0000 7f059be00700 1 -- 172.21.15.94:0/629894390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c19e340 con 0x7f059c07de40 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.357+0000 7f059be00700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c19de00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f058c011b60 tx=0x7f058c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.357+0000 7f0598c00700 1 -- 172.21.15.94:0/629894390 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f058c003970 con 0x7f059c07de40 2024-06-03T00:56:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.357+0000 7f0598c00700 1 -- 172.21.15.94:0/629894390 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f058c003b10 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.357+0000 7f0598c00700 1 -- 172.21.15.94:0/629894390 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f058c024610 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.357+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059c19e5d0 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.357+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059c1a12c0 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.358+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059c063af0 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.358+0000 7f0598c00700 1 -- 172.21.15.94:0/629894390 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f058c01a9f0 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.359+0000 7f0598c00700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f05800390a0 0x7f058003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.359+0000 7f0598c00700 1 -- 172.21.15.94:0/629894390 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f058c011e00 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.359+0000 7f059b400700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f05800390a0 0x7f058003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.360+0000 7f059b400700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f05800390a0 0x7f058003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0588006fd0 tx=0x7f0588006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.365+0000 7f0598c00700 1 -- 172.21.15.94:0/629894390 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f058c00a0d0 con 0x7f059c07de40 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.581+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f059c1044a0 con 0x7f05800390a0 2024-06-03T00:56:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.582+0000 7f0598c00700 1 -- 172.21.15.94:0/629894390 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f059c1044a0 con 0x7f05800390a0 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f05800390a0 msgr2=0x7f058003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f05800390a0 0x7f058003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0588006fd0 tx=0x7f0588006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 msgr2=0x7f059c19de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c19de00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f058c011b60 tx=0x7f058c011c40 comp rx=0 tx=0).stop 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 shutdown_connections 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f05800390a0 0x7f058003b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 --2- 172.21.15.94:0/629894390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f059c07de40 0x7f059c19de00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.584+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 >> 172.21.15.94:0/629894390 conn(0x7f059c1031d0 msgr2=0x7f059c103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.585+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 shutdown_connections 2024-06-03T00:56:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:03.585+0000 7f05a1400700 1 -- 172.21.15.94:0/629894390 wait complete. 2024-06-03T00:56:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-06-03T00:56:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:04 smithi094 ceph-mon[32119]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:05.458 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: Generating public/private rsa key pair. 2024-06-03T00:56:05.458 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: Your identification has been saved in /tmp/tmpql9sbbu_/key. 2024-06-03T00:56:05.458 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: Your public key has been saved in /tmp/tmpql9sbbu_/key.pub. 2024-06-03T00:56:05.458 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: The key fingerprint is: 2024-06-03T00:56:05.458 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: SHA256:qbrSwUa9/e6H/oYKUsnvH1fl3ce4SsgR3qg2MTXFKjo ceph-d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:56:05.458 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: The key's randomart image is: 2024-06-03T00:56:05.458 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: +---[RSA 3072]----+ 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | .. | 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | .. | 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | . +. .| 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | .....+.= =o| 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | o ++S.+ . o *| 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | +.Eo.= o . ..| 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | o..o.=.+oo . | 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | . .o + .++o. | 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: | .o. o*=+o | 2024-06-03T00:56:05.459 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: +----[SHA256]-----+ 2024-06-03T00:56:06.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.964+0000 7ff80ea00700 1 Processor -- start 2024-06-03T00:56:06.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.964+0000 7ff80ea00700 1 -- start start 2024-06-03T00:56:06.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.965+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80807cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:06.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.965+0000 7ff80ea00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff80807d100 con 0x7ff80807e6d0 2024-06-03T00:56:06.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.965+0000 7ff807e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80807cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.965+0000 7ff807e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80807cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38948/0 (socket says 172.21.15.94:38948) 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.965+0000 7ff807e00700 1 -- 172.21.15.94:0/67397031 learned_addr learned my addr 172.21.15.94:0/67397031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.966+0000 7ff807e00700 1 -- 172.21.15.94:0/67397031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80807d280 con 0x7ff80807e6d0 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.966+0000 7ff807e00700 1 --2- 172.21.15.94:0/67397031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80807cb30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80098a0 tx=0x7ff7f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f97afc200cadf33 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.967+0000 7ff806a00700 1 -- 172.21.15.94:0/67397031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f8004160 con 0x7ff80807e6d0 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.967+0000 7ff806a00700 1 -- 172.21.15.94:0/67397031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7f8004300 con 0x7ff80807e6d0 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.967+0000 7ff806a00700 1 -- 172.21.15.94:0/67397031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f8011b50 con 0x7ff80807e6d0 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.967+0000 7ff80ea00700 1 -- 172.21.15.94:0/67397031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 msgr2=0x7ff80807cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:06.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.967+0000 7ff80ea00700 1 --2- 172.21.15.94:0/67397031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80807cb30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80098a0 tx=0x7ff7f8009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.968+0000 7ff80ea00700 1 -- 172.21.15.94:0/67397031 shutdown_connections 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.968+0000 7ff80ea00700 1 --2- 172.21.15.94:0/67397031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80807cb30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.968+0000 7ff80ea00700 1 -- 172.21.15.94:0/67397031 >> 172.21.15.94:0/67397031 conn(0x7ff808103470 msgr2=0x7ff808105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.968+0000 7ff80ea00700 1 -- 172.21.15.94:0/67397031 shutdown_connections 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.968+0000 7ff80ea00700 1 -- 172.21.15.94:0/67397031 wait complete. 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.969+0000 7ff80ea00700 1 Processor -- start 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.969+0000 7ff80ea00700 1 -- start start 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.969+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.969+0000 7ff80ea00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff80810e420 con 0x7ff80807e6d0 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.970+0000 7ff807e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.970+0000 7ff807e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38960/0 (socket says 172.21.15.94:38960) 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.970+0000 7ff807e00700 1 -- 172.21.15.94:0/3367551407 learned_addr learned my addr 172.21.15.94:0/3367551407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:06.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.970+0000 7ff807e00700 1 -- 172.21.15.94:0/3367551407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80819e490 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.970+0000 7ff807e00700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80819df50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8011670 tx=0x7ff7f8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.971+0000 7ff804c00700 1 -- 172.21.15.94:0/3367551407 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f80039c0 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.971+0000 7ff804c00700 1 -- 172.21.15.94:0/3367551407 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7f8011e50 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.971+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80819e720 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.971+0000 7ff804c00700 1 -- 172.21.15.94:0/3367551407 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f801a5c0 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.971+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8081a1410 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.972+0000 7ff804c00700 1 -- 172.21.15.94:0/3367551407 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff7f8024430 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.972+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff808063af0 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.972+0000 7ff804c00700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7ff7ec039050 0x7ff7ec03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.972+0000 7ff804c00700 1 -- 172.21.15.94:0/3367551407 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff7f804d5e0 con 0x7ff80807e6d0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.973+0000 7ff807400700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7ff7ec039050 0x7ff7ec03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:06.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.973+0000 7ff807400700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7ff7ec039050 0x7ff7ec03b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4006fd0 tx=0x7ff7f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:04.978+0000 7ff804c00700 1 -- 172.21.15.94:0/3367551407 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff7f802d910 con 0x7ff80807e6d0 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.200+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff8081046e0 con 0x7ff7ec039050 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.434+0000 7ff804c00700 1 -- 172.21.15.94:0/3367551407 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8081046e0 con 0x7ff7ec039050 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.437+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7ff7ec039050 msgr2=0x7ff7ec03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.437+0000 7ff80ea00700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7ff7ec039050 0x7ff7ec03b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4006fd0 tx=0x7ff7f4006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 msgr2=0x7ff80819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80819df50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8011670 tx=0x7ff7f8011b60 comp rx=0 tx=0).stop 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 shutdown_connections 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7ff7ec039050 0x7ff7ec03b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 --2- 172.21.15.94:0/3367551407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80807e6d0 0x7ff80819df50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 >> 172.21.15.94:0/3367551407 conn(0x7ff808103470 msgr2=0x7ff808103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 shutdown_connections 2024-06-03T00:56:06.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:05.438+0000 7ff80ea00700 1 -- 172.21.15.94:0/3367551407 wait complete. 2024-06-03T00:56:06.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:06 smithi094 ceph-mon[32119]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:06.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:06 smithi094 ceph-mon[32119]: Generating ssh key... 2024-06-03T00:56:06.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:06 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:06.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:06 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:07.616 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpqtEOONP72NKuNBgkRlZzASXAWBNE4jgHwCgeq+GLW92oyuuqZ9XqqoLTob41afElXmOidYkkHs3uXlTBuJWBfm9IHQxhHjdQ6FzgFdFUcJSbFGUIMVtVRyK0wMvjH7YaUdbffnjyQL+w1RERWFygXA++jLv/CNnNpl5JyN3aRUX+41Cm834zDxOG4Mn5Fte2xpx+HUevIWmoFVYx+b7idbJwZPqsLzNqB2XlsXDis92xv8s/7raBUiupIkuFkT/C6ZY0uAfwS9jtvOKj9ml28ECKVxPRdmhMefBfJ8J+zMFvFQw9Ow7UWwkmg3gURMswx0sJs8rAVCmGDgcIYKJ7Y2ZDHfdRr90J+2MjEJ/OR3fGAVBrLqu11qMAlugYf2dtFGRSH/AcryDbaU+3wCFht8+cYKItPN6gyBEHWRlBuCFwLKz+PUCBYinwrxah31jEoWNmmsl0FCGGNe87jFPSENnNJNQBE2ZKS5ofhP4C20huxPWMZNk7UC23JW23yBc= ceph-d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:56:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.712+0000 7f7a15c00700 1 Processor -- start 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.712+0000 7f7a15c00700 1 -- start start 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.713+0000 7f7a15c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a100822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.713+0000 7f7a15c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10082870 con 0x7f7a10081e80 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.713+0000 7f7a0ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a100822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.713+0000 7f7a0ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a100822a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38970/0 (socket says 172.21.15.94:38970) 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.713+0000 7f7a0ea00700 1 -- 172.21.15.94:0/3300120400 learned_addr learned my addr 172.21.15.94:0/3300120400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.714+0000 7f7a0ea00700 1 -- 172.21.15.94:0/3300120400 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1010f620 con 0x7f7a10081e80 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.714+0000 7f7a0ea00700 1 --2- 172.21.15.94:0/3300120400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a100822a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a000098a0 tx=0x7f7a00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e363e47ddc96c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.715+0000 7f7a0d600700 1 -- 172.21.15.94:0/3300120400 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a00004160 con 0x7f7a10081e80 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.715+0000 7f7a0d600700 1 -- 172.21.15.94:0/3300120400 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a00004300 con 0x7f7a10081e80 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.715+0000 7f7a0d600700 1 -- 172.21.15.94:0/3300120400 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a00011b50 con 0x7f7a10081e80 2024-06-03T00:56:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.716+0000 7f7a15c00700 1 -- 172.21.15.94:0/3300120400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 msgr2=0x7f7a100822a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.716+0000 7f7a15c00700 1 --2- 172.21.15.94:0/3300120400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a100822a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a000098a0 tx=0x7f7a00009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.716+0000 7f7a15c00700 1 -- 172.21.15.94:0/3300120400 shutdown_connections 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.716+0000 7f7a15c00700 1 --2- 172.21.15.94:0/3300120400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a100822a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.716+0000 7f7a15c00700 1 -- 172.21.15.94:0/3300120400 >> 172.21.15.94:0/3300120400 conn(0x7f7a1007d520 msgr2=0x7f7a1007f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.716+0000 7f7a15c00700 1 -- 172.21.15.94:0/3300120400 shutdown_connections 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.716+0000 7f7a15c00700 1 -- 172.21.15.94:0/3300120400 wait complete. 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.717+0000 7f7a15c00700 1 Processor -- start 2024-06-03T00:56:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.717+0000 7f7a15c00700 1 -- start start 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.717+0000 7f7a15c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a101a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.717+0000 7f7a15c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101129d0 con 0x7f7a10081e80 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.718+0000 7f7a0ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a101a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.718+0000 7f7a0ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a101a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38976/0 (socket says 172.21.15.94:38976) 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.718+0000 7f7a0ea00700 1 -- 172.21.15.94:0/1383053123 learned_addr learned my addr 172.21.15.94:0/1383053123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.718+0000 7f7a0ea00700 1 -- 172.21.15.94:0/1383053123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a101a2910 con 0x7f7a10081e80 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.718+0000 7f7a0ea00700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a101a23d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000c00 tx=0x7f7a00011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.719+0000 7f7a07400700 1 -- 172.21.15.94:0/1383053123 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a000039e0 con 0x7f7a10081e80 2024-06-03T00:56:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.719+0000 7f7a07400700 1 -- 172.21.15.94:0/1383053123 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a00003b80 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.719+0000 7f7a07400700 1 -- 172.21.15.94:0/1383053123 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a00024820 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.719+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a101a2ba0 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.719+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101a5890 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.720+0000 7f7a07400700 1 -- 172.21.15.94:0/1383053123 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7a00022020 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.720+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a10063af0 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.720+0000 7f7a07400700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f79f40390a0 0x7f79f403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.721+0000 7f7a07400700 1 -- 172.21.15.94:0/1383053123 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a00011e00 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.721+0000 7f7a0e000700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f79f40390a0 0x7f79f403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.721+0000 7f7a0e000700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f79f40390a0 0x7f79f403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79fc006fd0 tx=0x7f79fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.726+0000 7f7a07400700 1 -- 172.21.15.94:0/1383053123 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7a00016260 con 0x7f7a10081e80 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.949+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7a1007f8a0 con 0x7f79f40390a0 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.951+0000 7f7a07400700 1 -- 172.21.15.94:0/1383053123 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7a1007f8a0 con 0x7f79f40390a0 2024-06-03T00:56:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f79f40390a0 msgr2=0x7f79f403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f79f40390a0 0x7f79f403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79fc006fd0 tx=0x7f79fc006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 msgr2=0x7f7a101a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a101a23d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000c00 tx=0x7f7a00011c10 comp rx=0 tx=0).stop 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 shutdown_connections 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f79f40390a0 0x7f79f403b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 --2- 172.21.15.94:0/1383053123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a10081e80 0x7f7a101a23d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.954+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 >> 172.21.15.94:0/1383053123 conn(0x7f7a1007d520 msgr2=0x7f7a1007f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.955+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 shutdown_connections 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:06.955+0000 7f7a15c00700 1 -- 172.21.15.94:0/1383053123 wait complete. 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-03T00:56:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-06-03T00:56:07.627 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-06-03T00:56:07.930 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:07 smithi094 ceph-mon[32119]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:08.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:08 smithi094 ceph-mon[32119]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:11.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:10 smithi094 ceph-mon[32119]: Deploying cephadm binary to smithi094 2024-06-03T00:56:15.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-06-03T00:56:15.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.240+0000 7fe1e9e00700 1 Processor -- start 2024-06-03T00:56:15.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.240+0000 7fe1e9e00700 1 -- start start 2024-06-03T00:56:15.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.241+0000 7fe1e9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e4108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:15.415 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.241+0000 7fe1e9e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e41085e0 con 0x7fe1e4107bf0 2024-06-03T00:56:15.415 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.241+0000 7fe1e8a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e4108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:15.415 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.241+0000 7fe1e8a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e4108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38986/0 (socket says 172.21.15.94:38986) 2024-06-03T00:56:15.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.241+0000 7fe1e8a00700 1 -- 172.21.15.94:0/3007338904 learned_addr learned my addr 172.21.15.94:0/3007338904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:15.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.242+0000 7fe1e8a00700 1 -- 172.21.15.94:0/3007338904 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e4108df0 con 0x7fe1e4107bf0 2024-06-03T00:56:15.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.242+0000 7fe1e8a00700 1 --2- 172.21.15.94:0/3007338904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e4108010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098a0 tx=0x7fe1d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85daa49f93a85d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:15.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e3400700 1 -- 172.21.15.94:0/3007338904 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d8004160 con 0x7fe1e4107bf0 2024-06-03T00:56:15.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e3400700 1 -- 172.21.15.94:0/3007338904 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1d8004300 con 0x7fe1e4107bf0 2024-06-03T00:56:15.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e3400700 1 -- 172.21.15.94:0/3007338904 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d8011b50 con 0x7fe1e4107bf0 2024-06-03T00:56:15.416 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e9e00700 1 -- 172.21.15.94:0/3007338904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 msgr2=0x7fe1e4108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e9e00700 1 --2- 172.21.15.94:0/3007338904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e4108010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098a0 tx=0x7fe1d8009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e9e00700 1 -- 172.21.15.94:0/3007338904 shutdown_connections 2024-06-03T00:56:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e9e00700 1 --2- 172.21.15.94:0/3007338904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e4108010 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.243+0000 7fe1e9e00700 1 -- 172.21.15.94:0/3007338904 >> 172.21.15.94:0/3007338904 conn(0x7fe1e41031d0 msgr2=0x7fe1e41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.244+0000 7fe1e9e00700 1 -- 172.21.15.94:0/3007338904 shutdown_connections 2024-06-03T00:56:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.244+0000 7fe1e9e00700 1 -- 172.21.15.94:0/3007338904 wait complete. 2024-06-03T00:56:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.245+0000 7fe1e9e00700 1 Processor -- start 2024-06-03T00:56:15.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.245+0000 7fe1e9e00700 1 -- start start 2024-06-03T00:56:15.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.245+0000 7fe1e9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:15.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.245+0000 7fe1e9e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e410c1a0 con 0x7fe1e4107bf0 2024-06-03T00:56:15.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.245+0000 7fe1e8a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:15.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.245+0000 7fe1e8a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39000/0 (socket says 172.21.15.94:39000) 2024-06-03T00:56:15.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.245+0000 7fe1e8a00700 1 -- 172.21.15.94:0/466735912 learned_addr learned my addr 172.21.15.94:0/466735912 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:15.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.246+0000 7fe1e8a00700 1 -- 172.21.15.94:0/466735912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e419c1b0 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.246+0000 7fe1e8a00700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e419bc70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8011850 tx=0x7fe1d8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.246+0000 7fe1e1600700 1 -- 172.21.15.94:0/466735912 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d8003af0 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.246+0000 7fe1e1600700 1 -- 172.21.15.94:0/466735912 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1d8020070 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.246+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e419c440 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.246+0000 7fe1e1600700 1 -- 172.21.15.94:0/466735912 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80195a0 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.247+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e419f130 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.248+0000 7fe1e1600700 1 -- 172.21.15.94:0/466735912 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe1d8019850 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.248+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e4063af0 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.248+0000 7fe1e1600700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fe1cc0390a0 0x7fe1cc03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.248+0000 7fe1e1600700 1 -- 172.21.15.94:0/466735912 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1d8011e00 con 0x7fe1e4107bf0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.249+0000 7fe1e3e00700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fe1cc0390a0 0x7fe1cc03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.249+0000 7fe1e3e00700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fe1cc0390a0 0x7fe1cc03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0006fd0 tx=0x7fe1d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:15.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.254+0000 7fe1e1600700 1 -- 172.21.15.94:0/466735912 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe1d800e260 con 0x7fe1e4107bf0 2024-06-03T00:56:15.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:08.477+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}) -- 0x7fe1e41044b0 con 0x7fe1cc0390a0 2024-06-03T00:56:15.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.713+0000 7fe1e1600700 1 -- 172.21.15.94:0/466735912 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe1e41044b0 con 0x7fe1cc0390a0 2024-06-03T00:56:15.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fe1cc0390a0 msgr2=0x7fe1cc03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:15.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fe1cc0390a0 0x7fe1cc03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0006fd0 tx=0x7fe1d0006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:15.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 msgr2=0x7fe1e419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:15.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e419bc70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8011850 tx=0x7fe1d8011c10 comp rx=0 tx=0).stop 2024-06-03T00:56:15.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 shutdown_connections 2024-06-03T00:56:15.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fe1cc0390a0 0x7fe1cc03b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:15.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 --2- 172.21.15.94:0/466735912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e4107bf0 0x7fe1e419bc70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:15.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 >> 172.21.15.94:0/466735912 conn(0x7fe1e41031d0 msgr2=0x7fe1e4103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:15.425 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.717+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 shutdown_connections 2024-06-03T00:56:15.425 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:14.718+0000 7fe1e9e00700 1 -- 172.21.15.94:0/466735912 wait complete. 2024-06-03T00:56:15.425 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mon service... 2024-06-03T00:56:15.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:15 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:15.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:15 smithi094 ceph-mon[32119]: Added host smithi094 2024-06-03T00:56:15.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:15 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:17.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:17 smithi094 ceph-mon[32119]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:17.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:17 smithi094 ceph-mon[32119]: Saving service mon spec with placement count:5 2024-06-03T00:56:17.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:17 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:17.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-03T00:56:17.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.127+0000 7f198c800700 1 Processor -- start 2024-06-03T00:56:17.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.127+0000 7f198c800700 1 -- start start 2024-06-03T00:56:17.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.128+0000 7f198c800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19840822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:17.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.128+0000 7f198c800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1984082870 con 0x7f1984081e80 2024-06-03T00:56:17.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.128+0000 7f1989e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19840822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.129+0000 7f1989e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19840822a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56584/0 (socket says 172.21.15.94:56584) 2024-06-03T00:56:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.129+0000 7f1989e00700 1 -- 172.21.15.94:0/1491464625 learned_addr learned my addr 172.21.15.94:0/1491464625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.129+0000 7f1989e00700 1 -- 172.21.15.94:0/1491464625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198410f620 con 0x7f1984081e80 2024-06-03T00:56:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.129+0000 7f1989e00700 1 --2- 172.21.15.94:0/1491464625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19840822a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19780098a0 tx=0x7f1978009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519fd64fe2e03d73 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:17.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.130+0000 7f1988a00700 1 -- 172.21.15.94:0/1491464625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1978004160 con 0x7f1984081e80 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.130+0000 7f1988a00700 1 -- 172.21.15.94:0/1491464625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1978004300 con 0x7f1984081e80 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.130+0000 7f1988a00700 1 -- 172.21.15.94:0/1491464625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1978011b50 con 0x7f1984081e80 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.130+0000 7f198c800700 1 -- 172.21.15.94:0/1491464625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 msgr2=0x7f19840822a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.130+0000 7f198c800700 1 --2- 172.21.15.94:0/1491464625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19840822a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19780098a0 tx=0x7f1978009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.131+0000 7f198c800700 1 -- 172.21.15.94:0/1491464625 shutdown_connections 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.131+0000 7f198c800700 1 --2- 172.21.15.94:0/1491464625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19840822a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.131+0000 7f198c800700 1 -- 172.21.15.94:0/1491464625 >> 172.21.15.94:0/1491464625 conn(0x7f198407d520 msgr2=0x7f198407f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:17.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.131+0000 7f198c800700 1 -- 172.21.15.94:0/1491464625 shutdown_connections 2024-06-03T00:56:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.131+0000 7f198c800700 1 -- 172.21.15.94:0/1491464625 wait complete. 2024-06-03T00:56:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.132+0000 7f198c800700 1 Processor -- start 2024-06-03T00:56:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.132+0000 7f198c800700 1 -- start start 2024-06-03T00:56:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.132+0000 7f198c800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19841a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:17.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.132+0000 7f198c800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19841129d0 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.133+0000 7f1989e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19841a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.133+0000 7f1989e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19841a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56598/0 (socket says 172.21.15.94:56598) 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.133+0000 7f1989e00700 1 -- 172.21.15.94:0/2523160643 learned_addr learned my addr 172.21.15.94:0/2523160643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.133+0000 7f1989e00700 1 -- 172.21.15.94:0/2523160643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19841a2910 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.133+0000 7f1989e00700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19841a23d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1978000c00 tx=0x7f1978011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.133+0000 7f197ea00700 1 -- 172.21.15.94:0/2523160643 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19780039e0 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.134+0000 7f197ea00700 1 -- 172.21.15.94:0/2523160643 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1978003b80 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.134+0000 7f197ea00700 1 -- 172.21.15.94:0/2523160643 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1978024820 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.134+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19841a2ba0 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.134+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19841a5890 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.135+0000 7f197ea00700 1 -- 172.21.15.94:0/2523160643 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1978022020 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.135+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1984063af0 con 0x7f1984081e80 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.135+0000 7f197ea00700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f196c0390a0 0x7f196c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:17.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.135+0000 7f197ea00700 1 -- 172.21.15.94:0/2523160643 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1978011e00 con 0x7f1984081e80 2024-06-03T00:56:17.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.136+0000 7f1989400700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f196c0390a0 0x7f196c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:17.524 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.136+0000 7f1989400700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f196c0390a0 0x7f196c03b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1974006fd0 tx=0x7f1974006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:17.524 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.141+0000 7f197ea00700 1 -- 172.21.15.94:0/2523160643 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1978019bf0 con 0x7f1984081e80 2024-06-03T00:56:17.524 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.385+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f198407f8a0 con 0x7f196c0390a0 2024-06-03T00:56:17.524 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.392+0000 7f197ea00700 1 -- 172.21.15.94:0/2523160643 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f198407f8a0 con 0x7f196c0390a0 2024-06-03T00:56:17.524 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f196c0390a0 msgr2=0x7f196c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:17.524 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f196c0390a0 0x7f196c03b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1974006fd0 tx=0x7f1974006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 msgr2=0x7f19841a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19841a23d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1978000c00 tx=0x7f1978011c10 comp rx=0 tx=0).stop 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 shutdown_connections 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f196c0390a0 0x7f196c03b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 --2- 172.21.15.94:0/2523160643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1984081e80 0x7f19841a23d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 >> 172.21.15.94:0/2523160643 conn(0x7f198407d520 msgr2=0x7f198407f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 shutdown_connections 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:16.395+0000 7f198c800700 1 -- 172.21.15.94:0/2523160643 wait complete. 2024-06-03T00:56:17.525 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mgr service... 2024-06-03T00:56:19.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:19 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:19.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:19 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:19.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:19 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.759+0000 7f8a06600700 1 Processor -- start 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.759+0000 7f8a06600700 1 -- start start 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.760+0000 7f8a06600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a00108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.760+0000 7f8a06600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001087d0 con 0x7f8a00107de0 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.760+0000 7f89ff400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a00108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.760+0000 7f89ff400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a00108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56604/0 (socket says 172.21.15.94:56604) 2024-06-03T00:56:19.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.760+0000 7f89ff400700 1 -- 172.21.15.94:0/2924937791 learned_addr learned my addr 172.21.15.94:0/2924937791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.761+0000 7f89ff400700 1 -- 172.21.15.94:0/2924937791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a00108fe0 con 0x7f8a00107de0 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.761+0000 7f89ff400700 1 --2- 172.21.15.94:0/2924937791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a00108200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89f00098a0 tx=0x7f89f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dffbf0f2ab299d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.761+0000 7f89fe000700 1 -- 172.21.15.94:0/2924937791 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89f0004160 con 0x7f8a00107de0 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.761+0000 7f89fe000700 1 -- 172.21.15.94:0/2924937791 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f0004300 con 0x7f8a00107de0 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.761+0000 7f89fe000700 1 -- 172.21.15.94:0/2924937791 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89f0011b20 con 0x7f8a00107de0 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.762+0000 7f8a06600700 1 -- 172.21.15.94:0/2924937791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 msgr2=0x7f8a00108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.762+0000 7f8a06600700 1 --2- 172.21.15.94:0/2924937791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a00108200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89f00098a0 tx=0x7f89f0009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.762+0000 7f8a06600700 1 -- 172.21.15.94:0/2924937791 shutdown_connections 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.762+0000 7f8a06600700 1 --2- 172.21.15.94:0/2924937791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a00108200 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.762+0000 7f8a06600700 1 -- 172.21.15.94:0/2924937791 >> 172.21.15.94:0/2924937791 conn(0x7f8a001033e0 msgr2=0x7f8a00105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.763+0000 7f8a06600700 1 -- 172.21.15.94:0/2924937791 shutdown_connections 2024-06-03T00:56:19.794 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.763+0000 7f8a06600700 1 -- 172.21.15.94:0/2924937791 wait complete. 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.764+0000 7f8a06600700 1 Processor -- start 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.764+0000 7f8a06600700 1 -- start start 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.764+0000 7f8a06600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a0019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.764+0000 7f8a06600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0010c390 con 0x7f8a00107de0 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.764+0000 7f89ff400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a0019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.764+0000 7f89ff400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a0019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56620/0 (socket says 172.21.15.94:56620) 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.764+0000 7f89ff400700 1 -- 172.21.15.94:0/3559078811 learned_addr learned my addr 172.21.15.94:0/3559078811 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.765+0000 7f89ff400700 1 -- 172.21.15.94:0/3559078811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0019e450 con 0x7f8a00107de0 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.765+0000 7f89ff400700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a0019df10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89f00117e0 tx=0x7f89f0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.765+0000 7f89f7e00700 1 -- 172.21.15.94:0/3559078811 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89f0003bf0 con 0x7f8a00107de0 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.765+0000 7f89f7e00700 1 -- 172.21.15.94:0/3559078811 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f001a910 con 0x7f8a00107de0 2024-06-03T00:56:19.795 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.765+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0019e6e0 con 0x7f8a00107de0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.765+0000 7f89f7e00700 1 -- 172.21.15.94:0/3559078811 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89f0024970 con 0x7f8a00107de0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.766+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0019eb40 con 0x7f8a00107de0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.767+0000 7f89f7e00700 1 -- 172.21.15.94:0/3559078811 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f89f0022020 con 0x7f8a00107de0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.767+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a00063af0 con 0x7f8a00107de0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.767+0000 7f89f7e00700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f89e40390a0 0x7f89e403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.768+0000 7f89f7e00700 1 -- 172.21.15.94:0/3559078811 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f89f004d900 con 0x7f8a00107de0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.768+0000 7f89fea00700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f89e40390a0 0x7f89e403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.768+0000 7f89fea00700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f89e40390a0 0x7f89e403b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:18.773+0000 7f89f7e00700 1 -- 172.21.15.94:0/3559078811 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f89f002cdf0 con 0x7f8a00107de0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.015+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8a001044e0 con 0x7f89e40390a0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.021+0000 7f89f7e00700 1 -- 172.21.15.94:0/3559078811 <== mgr.14120 v2:172.21.15.94:6800/127652834 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8a001044e0 con 0x7f89e40390a0 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f89e40390a0 msgr2=0x7f89e403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:19.796 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f89e40390a0 0x7f89e403b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 msgr2=0x7f8a0019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a0019df10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89f00117e0 tx=0x7f89f0011ba0 comp rx=0 tx=0).stop 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 shutdown_connections 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f89e40390a0 0x7f89e403b560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 --2- 172.21.15.94:0/3559078811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00107de0 0x7f8a0019df10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 >> 172.21.15.94:0/3559078811 conn(0x7f8a001033e0 msgr2=0x7f8a00103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.025+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 shutdown_connections 2024-06-03T00:56:19.797 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:19.026+0000 7f8a06600700 1 -- 172.21.15.94:0/3559078811 wait complete. 2024-06-03T00:56:20.628 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:20 smithi094 ceph-mon[32119]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:20.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:20 smithi094 ceph-mon[32119]: Saving service mgr spec with placement count:2 2024-06-03T00:56:20.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:20 smithi094 ceph-mon[32119]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:56:21.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.380+0000 7f9981200700 1 Processor -- start 2024-06-03T00:56:21.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.380+0000 7f9981200700 1 -- start start 2024-06-03T00:56:21.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.381+0000 7f9981200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c10ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:21.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.381+0000 7f9981200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f997c07bdf0 con 0x7f997c108920 2024-06-03T00:56:21.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.381+0000 7f9979e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c10ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:21.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.381+0000 7f9979e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c10ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56632/0 (socket says 172.21.15.94:56632) 2024-06-03T00:56:21.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.381+0000 7f9979e00700 1 -- 172.21.15.94:0/28991497 learned_addr learned my addr 172.21.15.94:0/28991497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:21.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.382+0000 7f9979e00700 1 -- 172.21.15.94:0/28991497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c10b250 con 0x7f997c108920 2024-06-03T00:56:21.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.382+0000 7f9979e00700 1 --2- 172.21.15.94:0/28991497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c10ad10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f996c0098a0 tx=0x7f996c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb9a1de158119be server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:21.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.382+0000 7f9978a00700 1 -- 172.21.15.94:0/28991497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f996c004160 con 0x7f997c108920 2024-06-03T00:56:21.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.382+0000 7f9978a00700 1 -- 172.21.15.94:0/28991497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f996c004300 con 0x7f997c108920 2024-06-03T00:56:21.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.382+0000 7f9978a00700 1 -- 172.21.15.94:0/28991497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f996c011b20 con 0x7f997c108920 2024-06-03T00:56:21.863 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.383+0000 7f9981200700 1 -- 172.21.15.94:0/28991497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 msgr2=0x7f997c10ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:21.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.383+0000 7f9981200700 1 --2- 172.21.15.94:0/28991497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c10ad10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f996c0098a0 tx=0x7f996c009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:21.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.383+0000 7f9981200700 1 -- 172.21.15.94:0/28991497 shutdown_connections 2024-06-03T00:56:21.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.383+0000 7f9981200700 1 --2- 172.21.15.94:0/28991497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c10ad10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:21.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.383+0000 7f9981200700 1 -- 172.21.15.94:0/28991497 >> 172.21.15.94:0/28991497 conn(0x7f997c1022d0 msgr2=0x7f997c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:21.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.383+0000 7f9981200700 1 -- 172.21.15.94:0/28991497 shutdown_connections 2024-06-03T00:56:21.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.383+0000 7f9981200700 1 -- 172.21.15.94:0/28991497 wait complete. 2024-06-03T00:56:21.864 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.384+0000 7f9981200700 1 Processor -- start 2024-06-03T00:56:21.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.385+0000 7f9981200700 1 -- start start 2024-06-03T00:56:21.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.385+0000 7f9981200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:21.865 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.385+0000 7f9981200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f997c10c250 con 0x7f997c108920 2024-06-03T00:56:21.867 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.385+0000 7f9979e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:21.867 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.385+0000 7f9979e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56640/0 (socket says 172.21.15.94:56640) 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.385+0000 7f9979e00700 1 -- 172.21.15.94:0/1220908385 learned_addr learned my addr 172.21.15.94:0/1220908385 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.386+0000 7f9979e00700 1 -- 172.21.15.94:0/1220908385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c19c250 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.386+0000 7f9979e00700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c19bd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f996c0117e0 tx=0x7f996c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.386+0000 7f9972a00700 1 -- 172.21.15.94:0/1220908385 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f996c003bf0 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.386+0000 7f9972a00700 1 -- 172.21.15.94:0/1220908385 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f996c01a910 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.386+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c19c4e0 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.386+0000 7f9972a00700 1 -- 172.21.15.94:0/1220908385 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f996c024970 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.386+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c19c940 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.387+0000 7f9972a00700 1 -- 172.21.15.94:0/1220908385 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f996c022020 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.387+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c063af0 con 0x7f997c108920 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.388+0000 7f9972a00700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f9960039050 0x7f996003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:21.868 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.388+0000 7f9972a00700 1 -- 172.21.15.94:0/1220908385 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f996c04d500 con 0x7f997c108920 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.388+0000 7f9979400700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f9960039050 0x7f996003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.389+0000 7f9979400700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f9960039050 0x7f996003b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9968006fd0 tx=0x7f9968006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.394+0000 7f9972a00700 1 -- 172.21.15.94:0/1220908385 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f996c011e20 con 0x7f997c108920 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.625+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f997c19f770 con 0x7f997c108920 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.628+0000 7f9972a00700 1 -- 172.21.15.94:0/1220908385 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f996c04ce90 con 0x7f997c108920 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f9960039050 msgr2=0x7f996003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f9960039050 0x7f996003b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9968006fd0 tx=0x7f9968006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:21.869 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 msgr2=0x7f997c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:21.870 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c19bd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f996c0117e0 tx=0x7f996c011c10 comp rx=0 tx=0).stop 2024-06-03T00:56:21.870 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 shutdown_connections 2024-06-03T00:56:21.870 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f9960039050 0x7f996003b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:21.870 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 --2- 172.21.15.94:0/1220908385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f997c108920 0x7f997c19bd10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:21.870 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 >> 172.21.15.94:0/1220908385 conn(0x7f997c1022d0 msgr2=0x7f997c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:21.870 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.631+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 shutdown_connections 2024-06-03T00:56:21.870 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:20.632+0000 7f9981200700 1 -- 172.21.15.94:0/1220908385 wait complete. 2024-06-03T00:56:22.024 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:21 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1220908385' entity='client.admin' 2024-06-03T00:56:22.025 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:21 smithi094 ceph-mon[32119]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:56:23.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:22 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:23.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:22 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/905476171' entity='client.admin' 2024-06-03T00:56:23.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.449+0000 7f1fcf200700 1 Processor -- start 2024-06-03T00:56:23.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.450+0000 7f1fcf200700 1 -- start start 2024-06-03T00:56:23.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.450+0000 7f1fcf200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:23.395 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.450+0000 7f1fcf200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc807c890 con 0x7f1fc807de60 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.451+0000 7f1fc7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.451+0000 7f1fc7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56646/0 (socket says 172.21.15.94:56646) 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.451+0000 7f1fc7e00700 1 -- 172.21.15.94:0/3123983845 learned_addr learned my addr 172.21.15.94:0/3123983845 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.451+0000 7f1fc7e00700 1 -- 172.21.15.94:0/3123983845 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc807ca10 con 0x7f1fc807de60 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.452+0000 7f1fc7e00700 1 --2- 172.21.15.94:0/3123983845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc807c2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80098a0 tx=0x7f1fb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dbf30720f48fdc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.452+0000 7f1fc6a00700 1 -- 172.21.15.94:0/3123983845 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fb8004160 con 0x7f1fc807de60 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.452+0000 7f1fc6a00700 1 -- 172.21.15.94:0/3123983845 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb8004300 con 0x7f1fc807de60 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.452+0000 7f1fc6a00700 1 -- 172.21.15.94:0/3123983845 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fb8011b20 con 0x7f1fc807de60 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.453+0000 7f1fcf200700 1 -- 172.21.15.94:0/3123983845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 msgr2=0x7f1fc807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:23.396 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.453+0000 7f1fcf200700 1 --2- 172.21.15.94:0/3123983845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc807c2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80098a0 tx=0x7f1fb8009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.453+0000 7f1fcf200700 1 -- 172.21.15.94:0/3123983845 shutdown_connections 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.453+0000 7f1fcf200700 1 --2- 172.21.15.94:0/3123983845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc807c2c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.453+0000 7f1fcf200700 1 -- 172.21.15.94:0/3123983845 >> 172.21.15.94:0/3123983845 conn(0x7f1fc81033e0 msgr2=0x7f1fc8105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.453+0000 7f1fcf200700 1 -- 172.21.15.94:0/3123983845 shutdown_connections 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.453+0000 7f1fcf200700 1 -- 172.21.15.94:0/3123983845 wait complete. 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.454+0000 7f1fcf200700 1 Processor -- start 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.454+0000 7f1fcf200700 1 -- start start 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.455+0000 7f1fcf200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.455+0000 7f1fcf200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc810e440 con 0x7f1fc807de60 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.455+0000 7f1fc7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.455+0000 7f1fc7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc819def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56654/0 (socket says 172.21.15.94:56654) 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.455+0000 7f1fc7e00700 1 -- 172.21.15.94:0/905476171 learned_addr learned my addr 172.21.15.94:0/905476171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:23.397 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.455+0000 7f1fc7e00700 1 -- 172.21.15.94:0/905476171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc819e430 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.456+0000 7f1fc7e00700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc819def0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80117e0 tx=0x7f1fb8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.456+0000 7f1fc4c00700 1 -- 172.21.15.94:0/905476171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fb8003bf0 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.456+0000 7f1fc4c00700 1 -- 172.21.15.94:0/905476171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb801a910 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.456+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc819e6c0 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.456+0000 7f1fc4c00700 1 -- 172.21.15.94:0/905476171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fb8024970 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.456+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc819eb20 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.457+0000 7f1fc4c00700 1 -- 172.21.15.94:0/905476171 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1fb8022020 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.457+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc8063af0 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.458+0000 7f1fc4c00700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f1fac039050 0x7f1fac03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.458+0000 7f1fc4c00700 1 -- 172.21.15.94:0/905476171 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1fb804d920 con 0x7f1fc807de60 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.458+0000 7f1fc7400700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f1fac039050 0x7f1fac03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:23.398 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.459+0000 7f1fc7400700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f1fac039050 0x7f1fac03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4006fd0 tx=0x7f1fb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.463+0000 7f1fc4c00700 1 -- 172.21.15.94:0/905476171 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1fb802d910 con 0x7f1fc807de60 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.690+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f1fc81a14f0 con 0x7f1fc807de60 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.693+0000 7f1fc4c00700 1 -- 172.21.15.94:0/905476171 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f1fb802d3f0 con 0x7f1fc807de60 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f1fac039050 msgr2=0x7f1fac03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f1fac039050 0x7f1fac03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4006fd0 tx=0x7f1fb4006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 msgr2=0x7f1fc819def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc819def0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80117e0 tx=0x7f1fb8011ba0 comp rx=0 tx=0).stop 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 shutdown_connections 2024-06-03T00:56:23.399 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f1fac039050 0x7f1fac03b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:23.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 --2- 172.21.15.94:0/905476171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fc807de60 0x7f1fc819def0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:23.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.697+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 >> 172.21.15.94:0/905476171 conn(0x7f1fc81033e0 msgr2=0x7f1fc8103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:23.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.698+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 shutdown_connections 2024-06-03T00:56:23.400 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:22.698+0000 7f1fcf200700 1 -- 172.21.15.94:0/905476171 wait complete. 2024-06-03T00:56:23.400 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-06-03T00:56:24.311 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:23 smithi094 ceph-mon[32119]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:56:25.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:25 smithi094 ceph-mon[32119]: from='mgr.14120 172.21.15.94:0/372995982' entity='mgr.x' 2024-06-03T00:56:25.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:25 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3109622417' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-03T00:56:25.477 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: ignoring --setuser ceph since I am not root 2024-06-03T00:56:25.477 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: ignoring --setgroup ceph since I am not root 2024-06-03T00:56:25.477 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:25.407+0000 7fb68e9b8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T00:56:25.810 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:25.667+0000 7fb68e9b8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:56:25.810 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:25.808+0000 7fb68e9b8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T00:56:26.344 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:26 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3109622417' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-03T00:56:26.344 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:26 smithi094 ceph-mon[32119]: mgrmap e9: x(active, since 25s) 2024-06-03T00:56:26.344 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:26.214+0000 7fb68e9b8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T00:56:26.345 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:26.330+0000 7fb68e9b8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T00:56:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.006+0000 7f2df5200700 1 Processor -- start 2024-06-03T00:56:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.006+0000 7f2df5200700 1 -- start start 2024-06-03T00:56:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.006+0000 7f2df5200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.006+0000 7f2df5200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2df007c870 con 0x7f2df007de40 2024-06-03T00:56:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.006+0000 7f2dede00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.007+0000 7f2dede00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56660/0 (socket says 172.21.15.94:56660) 2024-06-03T00:56:26.358 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.007+0000 7f2dede00700 1 -- 172.21.15.94:0/1642947173 learned_addr learned my addr 172.21.15.94:0/1642947173 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.007+0000 7f2dede00700 1 -- 172.21.15.94:0/1642947173 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df007c9f0 con 0x7f2df007de40 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.007+0000 7f2dede00700 1 --2- 172.21.15.94:0/1642947173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df007c2a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2de00098a0 tx=0x7f2de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac6768bcc084596 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.008+0000 7f2deca00700 1 -- 172.21.15.94:0/1642947173 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de0004160 con 0x7f2df007de40 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.008+0000 7f2deca00700 1 -- 172.21.15.94:0/1642947173 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0004300 con 0x7f2df007de40 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.008+0000 7f2deca00700 1 -- 172.21.15.94:0/1642947173 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de0011b20 con 0x7f2df007de40 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.008+0000 7f2df5200700 1 -- 172.21.15.94:0/1642947173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 msgr2=0x7f2df007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.008+0000 7f2df5200700 1 --2- 172.21.15.94:0/1642947173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df007c2a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2de00098a0 tx=0x7f2de0009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.009+0000 7f2df5200700 1 -- 172.21.15.94:0/1642947173 shutdown_connections 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.009+0000 7f2df5200700 1 --2- 172.21.15.94:0/1642947173 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df007c2a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.009+0000 7f2df5200700 1 -- 172.21.15.94:0/1642947173 >> 172.21.15.94:0/1642947173 conn(0x7f2df01033e0 msgr2=0x7f2df0105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.009+0000 7f2df5200700 1 -- 172.21.15.94:0/1642947173 shutdown_connections 2024-06-03T00:56:26.359 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.009+0000 7f2df5200700 1 -- 172.21.15.94:0/1642947173 wait complete. 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.010+0000 7f2df5200700 1 Processor -- start 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.010+0000 7f2df5200700 1 -- start start 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.010+0000 7f2df5200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.010+0000 7f2df5200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010e460 con 0x7f2df007de40 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.011+0000 7f2dede00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.011+0000 7f2dede00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df019deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56662/0 (socket says 172.21.15.94:56662) 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.011+0000 7f2dede00700 1 -- 172.21.15.94:0/3109622417 learned_addr learned my addr 172.21.15.94:0/3109622417 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.011+0000 7f2dede00700 1 -- 172.21.15.94:0/3109622417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df019e3f0 con 0x7f2df007de40 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.011+0000 7f2dede00700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df019deb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2de00117e0 tx=0x7f2de0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:26.360 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.011+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de0003ba0 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.012+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de001a910 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.012+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2de00248f0 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.012+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df019e680 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.012+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df019eae0 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.013+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2de0022020 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.013+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df0063af0 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.013+0000 7f2de6a00700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f2dd40390a0 0x7f2dd403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.013+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2de004d4c0 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.014+0000 7f2ded400700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f2dd40390a0 0x7f2dd403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.014+0000 7f2ded400700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f2dd40390a0 0x7f2dd403b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc006fd0 tx=0x7f2ddc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.020+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2de000f2e0 con 0x7f2df007de40 2024-06-03T00:56:26.361 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:24.310+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f2df01a14b0 con 0x7f2df007de40 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.088+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f2de002d3f0 con 0x7f2df007de40 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.092+0000 7f2de6a00700 1 -- 172.21.15.94:0/3109622417 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f2de001a430 con 0x7f2df007de40 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f2dd40390a0 msgr2=0x7f2dd403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f2dd40390a0 0x7f2dd403b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc006fd0 tx=0x7f2ddc006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 msgr2=0x7f2df019deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df019deb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2de00117e0 tx=0x7f2de0011ba0 comp rx=0 tx=0).stop 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 shutdown_connections 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f2dd40390a0 0x7f2dd403b560 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc006fd0 tx=0x7f2ddc006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 --2- 172.21.15.94:0/3109622417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df007de40 0x7f2df019deb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.094+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 >> 172.21.15.94:0/3109622417 conn(0x7f2df01033e0 msgr2=0x7f2df0103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:26.362 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.095+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 shutdown_connections 2024-06-03T00:56:26.363 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:25.095+0000 7f2df5200700 1 -- 172.21.15.94:0/3109622417 wait complete. 2024-06-03T00:56:27.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:27 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/211391566' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-03T00:56:28.266 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:27.903+0000 7fb68e9b8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T00:56:28.266 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:28 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:28.138+0000 7fb68e9b8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T00:56:28.282 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:56:28.282 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-03T00:56:28.282 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-03T00:56:28.282 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-03T00:56:28.282 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-03T00:56:28.282 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.224+0000 7f38c2200700 1 Processor -- start 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.224+0000 7f38c2200700 1 -- start start 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.225+0000 7f38c2200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.225+0000 7f38c2200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc07c890 con 0x7f38bc07de60 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.225+0000 7f38c0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.225+0000 7f38c0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48228/0 (socket says 172.21.15.94:48228) 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.225+0000 7f38c0e00700 1 -- 172.21.15.94:0/2822290194 learned_addr learned my addr 172.21.15.94:0/2822290194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.226+0000 7f38c0e00700 1 -- 172.21.15.94:0/2822290194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc07ca10 con 0x7f38bc07de60 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.226+0000 7f38c0e00700 1 --2- 172.21.15.94:0/2822290194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc07c2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f38b00098a0 tx=0x7f38b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e87969f556ac91 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.226+0000 7f38bb400700 1 -- 172.21.15.94:0/2822290194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b0004160 con 0x7f38bc07de60 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.226+0000 7f38bb400700 1 -- 172.21.15.94:0/2822290194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b0004300 con 0x7f38bc07de60 2024-06-03T00:56:28.283 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.226+0000 7f38bb400700 1 -- 172.21.15.94:0/2822290194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b0011b20 con 0x7f38bc07de60 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.227+0000 7f38c2200700 1 -- 172.21.15.94:0/2822290194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 msgr2=0x7f38bc07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.227+0000 7f38c2200700 1 --2- 172.21.15.94:0/2822290194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc07c2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f38b00098a0 tx=0x7f38b0009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.227+0000 7f38c2200700 1 -- 172.21.15.94:0/2822290194 shutdown_connections 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.227+0000 7f38c2200700 1 --2- 172.21.15.94:0/2822290194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc07c2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.227+0000 7f38c2200700 1 -- 172.21.15.94:0/2822290194 >> 172.21.15.94:0/2822290194 conn(0x7f38bc1031d0 msgr2=0x7f38bc1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.227+0000 7f38c2200700 1 -- 172.21.15.94:0/2822290194 shutdown_connections 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.227+0000 7f38c2200700 1 -- 172.21.15.94:0/2822290194 wait complete. 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.228+0000 7f38c2200700 1 Processor -- start 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.228+0000 7f38c2200700 1 -- start start 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.229+0000 7f38c2200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.229+0000 7f38c2200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc10e250 con 0x7f38bc07de60 2024-06-03T00:56:28.284 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.229+0000 7f38c0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.229+0000 7f38c0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48230/0 (socket says 172.21.15.94:48230) 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.229+0000 7f38c0e00700 1 -- 172.21.15.94:0/211391566 learned_addr learned my addr 172.21.15.94:0/211391566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.229+0000 7f38c0e00700 1 -- 172.21.15.94:0/211391566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc19e320 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.229+0000 7f38c0e00700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc19dde0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.230+0000 7f38b9600700 1 -- 172.21.15.94:0/211391566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b0003b60 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.230+0000 7f38b9600700 1 -- 172.21.15.94:0/211391566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b001a910 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.230+0000 7f38b9600700 1 -- 172.21.15.94:0/211391566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38b0024930 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.230+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc19e5b0 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.230+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc19ea10 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.231+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc063af0 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.231+0000 7f38b9600700 1 -- 172.21.15.94:0/211391566 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f38b0022020 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.232+0000 7f38b9600700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 0x7f38a403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.232+0000 7f38b9600700 1 -- 172.21.15.94:0/211391566 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f38b004d1f0 con 0x7f38bc07de60 2024-06-03T00:56:28.285 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.232+0000 7f38bbe00700 1 -- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 msgr2=0x7f38a403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/127652834 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.232+0000 7f38bbe00700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 0x7f38a403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.237+0000 7f38b9600700 1 -- 172.21.15.94:0/211391566 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38b000f2e0 con 0x7f38bc07de60 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.433+0000 7f38bbe00700 1 -- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 msgr2=0x7f38a403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/127652834 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.433+0000 7f38bbe00700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 0x7f38a403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.513+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f38bc1a13c0 con 0x7f38bc07de60 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.514+0000 7f38b9600700 1 -- 172.21.15.94:0/211391566 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f38b001ab70 con 0x7f38bc07de60 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 msgr2=0x7f38a403b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 0x7f38a403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 msgr2=0x7f38bc19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc19dde0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0011c10 comp rx=0 tx=0).stop 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 shutdown_connections 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7f38a4039100 0x7f38a403b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 --2- 172.21.15.94:0/211391566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38bc07de60 0x7f38bc19dde0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:28.287 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 >> 172.21.15.94:0/211391566 conn(0x7f38bc1031d0 msgr2=0x7f38bc103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:28.287 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 shutdown_connections 2024-06-03T00:56:28.287 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:27.517+0000 7f38c2200700 1 -- 172.21.15.94:0/211391566 wait complete. 2024-06-03T00:56:28.287 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-06-03T00:56:28.287 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-06-03T00:56:28.595 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:28 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:28.365+0000 7fb68e9b8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T00:56:28.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:28 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:28.758+0000 7fb68e9b8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T00:56:29.142 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:28 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:28.886+0000 7fb68e9b8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T00:56:29.546 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:29.140+0000 7fb68e9b8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T00:56:29.546 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:29.390+0000 7fb68e9b8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:56:30.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:30.169+0000 7fb68e9b8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T00:56:31.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:30.713+0000 7fb68e9b8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T00:56:31.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:30.839+0000 7fb68e9b8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T00:56:31.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:31.099+0000 7fb68e9b8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T00:56:32.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:31.739+0000 7fb68e9b8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T00:56:32.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:31.862+0000 7fb68e9b8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T00:56:32.468 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:32.188+0000 7fb68e9b8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T00:56:32.468 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:32.375+0000 7fb68e9b8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T00:56:32.766 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:32.507+0000 7fb68e9b8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T00:56:32.766 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:32.669+0000 7fb68e9b8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T00:56:33.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:33.046+0000 7fb68e9b8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T00:56:34.099 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:33.825+0000 7fb68e9b8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T00:56:34.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T00:56:34.109+0000 7fb68e9b8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T00:56:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: Active manager daemon x restarted 2024-06-03T00:56:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: Activating manager daemon x 2024-06-03T00:56:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: osdmap e3: 0 total, 0 up, 0 in 2024-06-03T00:56:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: mgrmap e10: x(active, starting, since 0.00525558s) 2024-06-03T00:56:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:56:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T00:56:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T00:56:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T00:56:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T00:56:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:34 smithi094 ceph-mon[32119]: Manager daemon x is now available 2024-06-03T00:56:35.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:35.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T00:56:35.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T00:56:35.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:35 smithi094 ceph-mon[32119]: mgrmap e11: x(active, since 1.00997s) 2024-06-03T00:56:36.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-06-03T00:56:36.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-03T00:56:36.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-03T00:56:36.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-06-03T00:56:36.031 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.934+0000 7fabd1e00700 1 Processor -- start 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.934+0000 7fabd1e00700 1 -- start start 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.934+0000 7fabd1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.934+0000 7fabd1e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1087d0 con 0x7fabcc107de0 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.935+0000 7fabcaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.935+0000 7fabcaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48244/0 (socket says 172.21.15.94:48244) 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.935+0000 7fabcaa00700 1 -- 172.21.15.94:0/3186649763 learned_addr learned my addr 172.21.15.94:0/3186649763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.935+0000 7fabcaa00700 1 -- 172.21.15.94:0/3186649763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc108fe0 con 0x7fabcc107de0 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.935+0000 7fabcaa00700 1 --2- 172.21.15.94:0/3186649763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc108200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0098a0 tx=0x7fabbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38645b3d27b3956c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.936+0000 7fabc9600700 1 -- 172.21.15.94:0/3186649763 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabbc004160 con 0x7fabcc107de0 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.936+0000 7fabc9600700 1 -- 172.21.15.94:0/3186649763 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabbc004300 con 0x7fabcc107de0 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.936+0000 7fabc9600700 1 -- 172.21.15.94:0/3186649763 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabbc011b20 con 0x7fabcc107de0 2024-06-03T00:56:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.936+0000 7fabd1e00700 1 -- 172.21.15.94:0/3186649763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 msgr2=0x7fabcc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.936+0000 7fabd1e00700 1 --2- 172.21.15.94:0/3186649763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc108200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0098a0 tx=0x7fabbc009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.937+0000 7fabd1e00700 1 -- 172.21.15.94:0/3186649763 shutdown_connections 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.937+0000 7fabd1e00700 1 --2- 172.21.15.94:0/3186649763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc108200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.937+0000 7fabd1e00700 1 -- 172.21.15.94:0/3186649763 >> 172.21.15.94:0/3186649763 conn(0x7fabcc1033e0 msgr2=0x7fabcc105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.937+0000 7fabd1e00700 1 -- 172.21.15.94:0/3186649763 shutdown_connections 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.937+0000 7fabd1e00700 1 -- 172.21.15.94:0/3186649763 wait complete. 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.938+0000 7fabd1e00700 1 Processor -- start 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.938+0000 7fabd1e00700 1 -- start start 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.939+0000 7fabd1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc07db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.939+0000 7fabd1e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc10c390 con 0x7fabcc107de0 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.939+0000 7fabcaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc07db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.939+0000 7fabcaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc07db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48254/0 (socket says 172.21.15.94:48254) 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.939+0000 7fabcaa00700 1 -- 172.21.15.94:0/1251301645 learned_addr learned my addr 172.21.15.94:0/1251301645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:36.033 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.939+0000 7fabcaa00700 1 -- 172.21.15.94:0/1251301645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc07e070 con 0x7fabcc107de0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.940+0000 7fabcaa00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc07db30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0117e0 tx=0x7fabbc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.940+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabbc003bf0 con 0x7fabcc107de0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.940+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabbc01a910 con 0x7fabcc107de0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.940+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabbc0249a0 con 0x7fabcc107de0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.940+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc07c260 con 0x7fabcc107de0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.940+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc07c720 con 0x7fabcc107de0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.941+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fabbc022020 con 0x7fabcc107de0 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.942+0000 7fabd0a00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:36.034 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.942+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 --> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fabb003bc80 con 0x7fabb00390b0 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.942+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fabbc04d390 con 0x7fabcc107de0 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.942+0000 7fabca000700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 msgr2=0x7fabb003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/127652834 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:28.942+0000 7fabca000700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:29.143+0000 7fabca000700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 msgr2=0x7fabb003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/127652834 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:29.143+0000 7fabca000700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:29.544+0000 7fabca000700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 msgr2=0x7fabb003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/127652834 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:29.544+0000 7fabca000700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:30.345+0000 7fabca000700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 msgr2=0x7fabb003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/127652834 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:30.345+0000 7fabca000700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:31.948+0000 7fabca000700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 msgr2=0x7fabb003b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/127652834 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:31.948+0000 7fabca000700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:34.116+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fabbc019ca0 con 0x7fabcc107de0 2024-06-03T00:56:36.035 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:34.116+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 msgr2=0x7fabb003b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:34.116+0000 7fabd0a00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.117+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fabbc04f390 con 0x7fabcc107de0 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.118+0000 7fabd0a00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fabb003cb70 0x7fabb003ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.118+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fabb003bc80 con 0x7fabb003cb70 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.119+0000 7fabca000700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fabb003cb70 0x7fabb003ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.121+0000 7fabca000700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fabb003cb70 0x7fabb003ef60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fabb8003a10 tx=0x7fabb8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.122+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fabb003bc80 con 0x7fabb003cb70 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.133+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fabcc063af0 con 0x7fabb003cb70 2024-06-03T00:56:36.036 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.133+0000 7fabd0a00700 1 -- 172.21.15.94:0/1251301645 <== mgr.14152 v2:172.21.15.94:6800/4120992823 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fabcc063af0 con 0x7fabb003cb70 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fabb003cb70 msgr2=0x7fabb003ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fabb003cb70 0x7fabb003ef60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fabb8003a10 tx=0x7fabb8009270 comp rx=0 tx=0).stop 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 msgr2=0x7fabcc07db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc07db30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0117e0 tx=0x7fabbc011c10 comp rx=0 tx=0).stop 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 shutdown_connections 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fabb003cb70 0x7fabb003ef60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fabcc107de0 0x7fabcc07db30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 --2- 172.21.15.94:0/1251301645 >> [v2:172.21.15.94:6800/127652834,v1:172.21.15.94:6801/127652834] conn(0x7fabb00390b0 0x7fabb003b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 >> 172.21.15.94:0/1251301645 conn(0x7fabcc1033e0 msgr2=0x7fabcc103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 shutdown_connections 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:35.134+0000 7fabd1e00700 1 -- 172.21.15.94:0/1251301645 wait complete. 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-06-03T00:56:36.037 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-06-03T00:56:36.277 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:36 smithi094 ceph-mon[32119]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-03T00:56:36.277 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:36 smithi094 ceph-mon[32119]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-03T00:56:37.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:37 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:35] ENGINE Bus STARTING 2024-06-03T00:56:37.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:37 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:35] ENGINE Serving on https://172.21.15.94:7150 2024-06-03T00:56:37.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:37 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:35] ENGINE Serving on http://172.21.15.94:8765 2024-06-03T00:56:37.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:37 smithi094 ceph-mon[32119]: [03/Jun/2024:00:56:35] ENGINE Bus STARTED 2024-06-03T00:56:37.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:37 smithi094 ceph-mon[32119]: mgrmap e12: x(active, since 2s) 2024-06-03T00:56:37.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.135+0000 7fb627c00700 1 Processor -- start 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.135+0000 7fb627c00700 1 -- start start 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.136+0000 7fb627c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62010a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.136+0000 7fb627c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010aa00 con 0x7fb62010a010 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.136+0000 7fb625200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.137+0000 7fb625200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34064/0 (socket says 172.21.15.94:34064) 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.137+0000 7fb625200700 1 -- 172.21.15.94:0/229471635 learned_addr learned my addr 172.21.15.94:0/229471635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:38.345 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.137+0000 7fb625200700 1 -- 172.21.15.94:0/229471635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62010b210 con 0x7fb62010a010 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.137+0000 7fb625200700 1 --2- 172.21.15.94:0/229471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62010a430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098a0 tx=0x7fb614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efd36f388522691 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.138+0000 7fb61f400700 1 -- 172.21.15.94:0/229471635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb614004160 con 0x7fb62010a010 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.138+0000 7fb61f400700 1 -- 172.21.15.94:0/229471635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb614004300 con 0x7fb62010a010 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.138+0000 7fb61f400700 1 -- 172.21.15.94:0/229471635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb614011b20 con 0x7fb62010a010 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.138+0000 7fb627c00700 1 -- 172.21.15.94:0/229471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 msgr2=0x7fb62010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.138+0000 7fb627c00700 1 --2- 172.21.15.94:0/229471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62010a430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6140098a0 tx=0x7fb614009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.139+0000 7fb627c00700 1 -- 172.21.15.94:0/229471635 shutdown_connections 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.139+0000 7fb627c00700 1 --2- 172.21.15.94:0/229471635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62010a430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.139+0000 7fb627c00700 1 -- 172.21.15.94:0/229471635 >> 172.21.15.94:0/229471635 conn(0x7fb62007d3e0 msgr2=0x7fb62007d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.139+0000 7fb627c00700 1 -- 172.21.15.94:0/229471635 shutdown_connections 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.139+0000 7fb627c00700 1 -- 172.21.15.94:0/229471635 wait complete. 2024-06-03T00:56:38.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.140+0000 7fb627c00700 1 Processor -- start 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.141+0000 7fb627c00700 1 -- start start 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.141+0000 7fb627c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.141+0000 7fb627c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010e5c0 con 0x7fb62010a010 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.141+0000 7fb625200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.141+0000 7fb625200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62019df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34076/0 (socket says 172.21.15.94:34076) 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.141+0000 7fb625200700 1 -- 172.21.15.94:0/815114912 learned_addr learned my addr 172.21.15.94:0/815114912 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.142+0000 7fb625200700 1 -- 172.21.15.94:0/815114912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62019e480 con 0x7fb62010a010 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.142+0000 7fb625200700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62019df40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb6140117e0 tx=0x7fb614011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.142+0000 7fb61d600700 1 -- 172.21.15.94:0/815114912 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb614003bf0 con 0x7fb62010a010 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.142+0000 7fb61d600700 1 -- 172.21.15.94:0/815114912 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61401a910 con 0x7fb62010a010 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.142+0000 7fb61d600700 1 -- 172.21.15.94:0/815114912 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb614024970 con 0x7fb62010a010 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.142+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62019e710 con 0x7fb62010a010 2024-06-03T00:56:38.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.143+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62019eb70 con 0x7fb62010a010 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.144+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb620063af0 con 0x7fb62010a010 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.144+0000 7fb61d600700 1 -- 172.21.15.94:0/815114912 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fb614022020 con 0x7fb62010a010 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.144+0000 7fb61d600700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb608039100 0x7fb60803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.144+0000 7fb61d600700 1 -- 172.21.15.94:0/815114912 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb61404d590 con 0x7fb62010a010 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.145+0000 7fb61fe00700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb608039100 0x7fb60803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.145+0000 7fb61fe00700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb608039100 0x7fb60803b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006fd0 tx=0x7fb60c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.150+0000 7fb61d600700 1 -- 172.21.15.94:0/815114912 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb6140212c0 con 0x7fb62010a010 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.380+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb620107b30 con 0x7fb608039100 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.586+0000 7fb61d600700 1 -- 172.21.15.94:0/815114912 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb620107b30 con 0x7fb608039100 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb608039100 msgr2=0x7fb60803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb608039100 0x7fb60803b5c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb60c006fd0 tx=0x7fb60c006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 msgr2=0x7fb62019df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:38.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62019df40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb6140117e0 tx=0x7fb614011ba0 comp rx=0 tx=0).stop 2024-06-03T00:56:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 shutdown_connections 2024-06-03T00:56:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb608039100 0x7fb60803b5c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 --2- 172.21.15.94:0/815114912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb62010a010 0x7fb62019df40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.589+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 >> 172.21.15.94:0/815114912 conn(0x7fb62007d3e0 msgr2=0x7fb620107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.590+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 shutdown_connections 2024-06-03T00:56:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:37.590+0000 7fb627c00700 1 -- 172.21.15.94:0/815114912 wait complete. 2024-06-03T00:56:38.349 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-06-03T00:56:38.836 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:38 smithi094 ceph-mon[32119]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:38.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:38 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:38.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:38 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:40.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:40.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Z7uwPo/eDqAl6ygsDlBjsOnwGZ5DwQKgUABvDEIvptnAAWcE78VT6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717376199, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-03T00:56:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.001+0000 7f2775e00700 1 Processor -- start 2024-06-03T00:56:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.001+0000 7f2775e00700 1 -- start start 2024-06-03T00:56:40.169 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.001+0000 7f2775e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:40.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.001+0000 7f2775e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f277007c7e0 con 0x7f277007dd30 2024-06-03T00:56:40.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.002+0000 7f2774a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:40.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.002+0000 7f2774a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34082/0 (socket says 172.21.15.94:34082) 2024-06-03T00:56:40.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.002+0000 7f2774a00700 1 -- 172.21.15.94:0/2930216038 learned_addr learned my addr 172.21.15.94:0/2930216038 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:40.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.002+0000 7f2774a00700 1 -- 172.21.15.94:0/2930216038 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277007c960 con 0x7f277007dd30 2024-06-03T00:56:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.002+0000 7f2774a00700 1 --2- 172.21.15.94:0/2930216038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277007c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27640098a0 tx=0x7f2764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff508692023d406 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.003+0000 7f276f400700 1 -- 172.21.15.94:0/2930216038 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764004160 con 0x7f277007dd30 2024-06-03T00:56:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.003+0000 7f276f400700 1 -- 172.21.15.94:0/2930216038 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2764004300 con 0x7f277007dd30 2024-06-03T00:56:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.003+0000 7f276f400700 1 -- 172.21.15.94:0/2930216038 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764011b20 con 0x7f277007dd30 2024-06-03T00:56:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.003+0000 7f2775e00700 1 -- 172.21.15.94:0/2930216038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 msgr2=0x7f277007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.003+0000 7f2775e00700 1 --2- 172.21.15.94:0/2930216038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277007c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27640098a0 tx=0x7f2764009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.004+0000 7f2775e00700 1 -- 172.21.15.94:0/2930216038 shutdown_connections 2024-06-03T00:56:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.004+0000 7f2775e00700 1 --2- 172.21.15.94:0/2930216038 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277007c2a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.004+0000 7f2775e00700 1 -- 172.21.15.94:0/2930216038 >> 172.21.15.94:0/2930216038 conn(0x7f27701032a0 msgr2=0x7f2770105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.004+0000 7f2775e00700 1 -- 172.21.15.94:0/2930216038 shutdown_connections 2024-06-03T00:56:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.004+0000 7f2775e00700 1 -- 172.21.15.94:0/2930216038 wait complete. 2024-06-03T00:56:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.005+0000 7f2775e00700 1 Processor -- start 2024-06-03T00:56:40.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.005+0000 7f2775e00700 1 -- start start 2024-06-03T00:56:40.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.005+0000 7f2775e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:40.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.005+0000 7f2775e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f277010e1b0 con 0x7f277007dd30 2024-06-03T00:56:40.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.005+0000 7f2774a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:40.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.006+0000 7f2774a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277019dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34086/0 (socket says 172.21.15.94:34086) 2024-06-03T00:56:40.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.006+0000 7f2774a00700 1 -- 172.21.15.94:0/343519852 learned_addr learned my addr 172.21.15.94:0/343519852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:40.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.006+0000 7f2774a00700 1 -- 172.21.15.94:0/343519852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277019e230 con 0x7f277007dd30 2024-06-03T00:56:40.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.006+0000 7f2774a00700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277019dcf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27640117e0 tx=0x7f2764011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:40.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.006+0000 7f276d600700 1 -- 172.21.15.94:0/343519852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764003ba0 con 0x7f277007dd30 2024-06-03T00:56:40.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.007+0000 7f276d600700 1 -- 172.21.15.94:0/343519852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276401a910 con 0x7f277007dd30 2024-06-03T00:56:40.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.007+0000 7f276d600700 1 -- 172.21.15.94:0/343519852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27640248f0 con 0x7f277007dd30 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.007+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277019e4c0 con 0x7f277007dd30 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.007+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277019e920 con 0x7f277007dd30 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.008+0000 7f276d600700 1 -- 172.21.15.94:0/343519852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f2764022020 con 0x7f277007dd30 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.008+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2770063af0 con 0x7f277007dd30 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.008+0000 7f276d600700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f27580390a0 0x7f275803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.009+0000 7f276d600700 1 -- 172.21.15.94:0/343519852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f276404dac0 con 0x7f277007dd30 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.009+0000 7f276fe00700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f27580390a0 0x7f275803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:40.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.009+0000 7f276fe00700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f27580390a0 0x7f275803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f275c006fd0 tx=0x7f275c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:40.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.015+0000 7f276d600700 1 -- 172.21.15.94:0/343519852 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f276400f2e0 con 0x7f277007dd30 2024-06-03T00:56:40.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.258+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f27701a1340 con 0x7f27580390a0 2024-06-03T00:56:40.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.498+0000 7f276d600700 1 -- 172.21.15.94:0/343519852 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f27701a1340 con 0x7f27580390a0 2024-06-03T00:56:40.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f27580390a0 msgr2=0x7f275803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:40.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f27580390a0 0x7f275803b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f275c006fd0 tx=0x7f275c006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:40.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 msgr2=0x7f277019dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:40.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277019dcf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27640117e0 tx=0x7f2764011ba0 comp rx=0 tx=0).stop 2024-06-03T00:56:40.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 shutdown_connections 2024-06-03T00:56:40.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f27580390a0 0x7f275803b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:40.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 --2- 172.21.15.94:0/343519852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f277007dd30 0x7f277019dcf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:40.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 >> 172.21.15.94:0/343519852 conn(0x7f27701032a0 msgr2=0x7f2770103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:40.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 shutdown_connections 2024-06-03T00:56:40.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:39.502+0000 7f2775e00700 1 -- 172.21.15.94:0/343519852 wait complete. 2024-06-03T00:56:40.177 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-06-03T00:56:40.984 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:40 smithi094 ceph-mon[32119]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:42.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:41 smithi094 ceph-mon[32119]: mgrmap e13: x(active, since 6s) 2024-06-03T00:56:42.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:41 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1422034373' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-03T00:56:42.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-06-03T00:56:42.240 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.942+0000 7fdd74600700 1 Processor -- start 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.942+0000 7fdd74600700 1 -- start start 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.942+0000 7fdd74600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.942+0000 7fdd74600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c07be50 con 0x7fdd6c1066b0 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.943+0000 7fdd71c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.943+0000 7fdd71c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34090/0 (socket says 172.21.15.94:34090) 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.943+0000 7fdd71c00700 1 -- 172.21.15.94:0/3616206961 learned_addr learned my addr 172.21.15.94:0/3616206961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.943+0000 7fdd71c00700 1 -- 172.21.15.94:0/3616206961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c107010 con 0x7fdd6c1066b0 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.943+0000 7fdd71c00700 1 --2- 172.21.15.94:0/3616206961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c106ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd600098a0 tx=0x7fdd60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c66aed6c1534b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.944+0000 7fdd6be00700 1 -- 172.21.15.94:0/3616206961 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd60004160 con 0x7fdd6c1066b0 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.944+0000 7fdd6be00700 1 -- 172.21.15.94:0/3616206961 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd60004300 con 0x7fdd6c1066b0 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.944+0000 7fdd6be00700 1 -- 172.21.15.94:0/3616206961 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd60011b20 con 0x7fdd6c1066b0 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.944+0000 7fdd74600700 1 -- 172.21.15.94:0/3616206961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 msgr2=0x7fdd6c106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:42.241 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.944+0000 7fdd74600700 1 --2- 172.21.15.94:0/3616206961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c106ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd600098a0 tx=0x7fdd60009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.945+0000 7fdd74600700 1 -- 172.21.15.94:0/3616206961 shutdown_connections 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.945+0000 7fdd74600700 1 --2- 172.21.15.94:0/3616206961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c106ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.945+0000 7fdd74600700 1 -- 172.21.15.94:0/3616206961 >> 172.21.15.94:0/3616206961 conn(0x7fdd6c1022d0 msgr2=0x7fdd6c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.945+0000 7fdd74600700 1 -- 172.21.15.94:0/3616206961 shutdown_connections 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.945+0000 7fdd74600700 1 -- 172.21.15.94:0/3616206961 wait complete. 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.946+0000 7fdd74600700 1 Processor -- start 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.946+0000 7fdd74600700 1 -- start start 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.946+0000 7fdd74600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.946+0000 7fdd74600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1080e0 con 0x7fdd6c1066b0 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.947+0000 7fdd71c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.947+0000 7fdd71c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34094/0 (socket says 172.21.15.94:34094) 2024-06-03T00:56:42.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.947+0000 7fdd71c00700 1 -- 172.21.15.94:0/1422034373 learned_addr learned my addr 172.21.15.94:0/1422034373 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.947+0000 7fdd71c00700 1 -- 172.21.15.94:0/1422034373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c19e440 con 0x7fdd6c1066b0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.947+0000 7fdd71c00700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c19df00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd600117e0 tx=0x7fdd60011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.947+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd60003bf0 con 0x7fdd6c1066b0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.948+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c19e6d0 con 0x7fdd6c1066b0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.948+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd6001a910 con 0x7fdd6c1066b0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.948+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd60024970 con 0x7fdd6c1066b0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.948+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c19eb30 con 0x7fdd6c1066b0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.949+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4c005360 con 0x7fdd6c1066b0 2024-06-03T00:56:42.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.949+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fdd60022020 con 0x7fdd6c1066b0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.950+0000 7fdd6a000700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdd540390b0 0x7fdd5403b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.950+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdd6004d470 con 0x7fdd6c1066b0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.950+0000 7fdd71200700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdd540390b0 0x7fdd5403b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.951+0000 7fdd71200700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdd540390b0 0x7fdd5403b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd58006fd0 tx=0x7fdd58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.956+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdd60011e20 con 0x7fdd6c1066b0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:40.962+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdd6001a430 con 0x7fdd6c1066b0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.182+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdd4c005650 con 0x7fdd6c1066b0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.183+0000 7fdd6a000700 1 -- 172.21.15.94:0/1422034373 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdd6000b3e0 con 0x7fdd6c1066b0 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.185+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdd540390b0 msgr2=0x7fdd5403b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:42.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.185+0000 7fdd74600700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdd540390b0 0x7fdd5403b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdd58006fd0 tx=0x7fdd58006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 msgr2=0x7fdd6c19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c19df00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd600117e0 tx=0x7fdd60011c10 comp rx=0 tx=0).stop 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 shutdown_connections 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdd540390b0 0x7fdd5403b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 --2- 172.21.15.94:0/1422034373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1066b0 0x7fdd6c19df00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 >> 172.21.15.94:0/1422034373 conn(0x7fdd6c1022d0 msgr2=0x7fdd6c194b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 shutdown_connections 2024-06-03T00:56:42.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:41.186+0000 7fdd74600700 1 -- 172.21.15.94:0/1422034373 wait complete. 2024-06-03T00:56:42.275 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-03T00:56:42.275 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-06-03T00:56:42.302 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-03T00:56:42.302 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-06-03T00:56:42.302 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-06-03T00:56:42.302 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-03T00:56:42.310 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-06-03T00:56:42.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:42.310 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-06-03T00:56:42.310 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-06-03T00:56:42.310 INFO:teuthology.orchestra.run.smithi094.stdout: Password: y4x1sqs31v 2024-06-03T00:56:42.310 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:42.311 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config directory 2024-06-03T00:56:42.313 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling autotune for osd_memory_target 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.896+0000 7f0aabe00700 1 Processor -- start 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.896+0000 7f0aabe00700 1 -- start start 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.897+0000 7f0aabe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa4108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.897+0000 7f0aabe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41087d0 con 0x7f0aa4107de0 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.897+0000 7f0aa9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa4108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.897+0000 7f0aa9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa4108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34104/0 (socket says 172.21.15.94:34104) 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.897+0000 7f0aa9400700 1 -- 172.21.15.94:0/1788293709 learned_addr learned my addr 172.21.15.94:0/1788293709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.898+0000 7f0aa9400700 1 -- 172.21.15.94:0/1788293709 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa4108fe0 con 0x7f0aa4107de0 2024-06-03T00:56:43.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.898+0000 7f0aa9400700 1 --2- 172.21.15.94:0/1788293709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa4108200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009e80 tx=0x7f0a9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebce2ab72977c1e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.898+0000 7f0aa3e00700 1 -- 172.21.15.94:0/1788293709 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a98017070 con 0x7f0aa4107de0 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.898+0000 7f0aa3e00700 1 -- 172.21.15.94:0/1788293709 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9800b8c0 con 0x7f0aa4107de0 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.898+0000 7f0aa3e00700 1 -- 172.21.15.94:0/1788293709 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a980045c0 con 0x7f0aa4107de0 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.899+0000 7f0aabe00700 1 -- 172.21.15.94:0/1788293709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 msgr2=0x7f0aa4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.899+0000 7f0aabe00700 1 --2- 172.21.15.94:0/1788293709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa4108200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009e80 tx=0x7f0a9800b330 comp rx=0 tx=0).stop 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.899+0000 7f0aabe00700 1 -- 172.21.15.94:0/1788293709 shutdown_connections 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.899+0000 7f0aabe00700 1 --2- 172.21.15.94:0/1788293709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa4108200 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.899+0000 7f0aabe00700 1 -- 172.21.15.94:0/1788293709 >> 172.21.15.94:0/1788293709 conn(0x7f0aa4103420 msgr2=0x7f0aa4105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.899+0000 7f0aabe00700 1 -- 172.21.15.94:0/1788293709 shutdown_connections 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.899+0000 7f0aabe00700 1 -- 172.21.15.94:0/1788293709 wait complete. 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.900+0000 7f0aabe00700 1 Processor -- start 2024-06-03T00:56:43.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.901+0000 7f0aabe00700 1 -- start start 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.901+0000 7f0aabe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.901+0000 7f0aabe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa410c390 con 0x7f0aa4107de0 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.901+0000 7f0aa9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.901+0000 7f0aa9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34106/0 (socket says 172.21.15.94:34106) 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.901+0000 7f0aa9400700 1 -- 172.21.15.94:0/3414115639 learned_addr learned my addr 172.21.15.94:0/3414115639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.902+0000 7f0aa9400700 1 -- 172.21.15.94:0/3414115639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa419e460 con 0x7f0aa4107de0 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.902+0000 7f0aa9400700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa419df20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009fd0 tx=0x7f0a98003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.902+0000 7f0aa2000700 1 -- 172.21.15.94:0/3414115639 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a98020020 con 0x7f0aa4107de0 2024-06-03T00:56:43.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.902+0000 7f0aa2000700 1 -- 172.21.15.94:0/3414115639 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9800bba0 con 0x7f0aa4107de0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.902+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa419e6f0 con 0x7f0aa4107de0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.902+0000 7f0aa2000700 1 -- 172.21.15.94:0/3414115639 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a98017040 con 0x7f0aa4107de0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.902+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa41a13e0 con 0x7f0aa4107de0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.904+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa4063af0 con 0x7f0aa4107de0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.904+0000 7f0aa2000700 1 -- 172.21.15.94:0/3414115639 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0a98023020 con 0x7f0aa4107de0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.904+0000 7f0aa2000700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0a8c039140 0x7f0a8c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.904+0000 7f0aa2000700 1 -- 172.21.15.94:0/3414115639 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a9800f050 con 0x7f0aa4107de0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.904+0000 7f0aa8a00700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0a8c039140 0x7f0a8c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:43.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.905+0000 7f0aa8a00700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0a8c039140 0x7f0a8c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006fd0 tx=0x7f0a90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:42.910+0000 7f0aa2000700 1 -- 172.21.15.94:0/3414115639 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a9802a020 con 0x7f0aa4107de0 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.131+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f0aa41a1a20 con 0x7f0aa4107de0 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.132+0000 7f0aa2000700 1 -- 172.21.15.94:0/3414115639 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f0a9801bbb0 con 0x7f0aa4107de0 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0a8c039140 msgr2=0x7f0a8c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0a8c039140 0x7f0a8c03b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006fd0 tx=0x7f0a90006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 msgr2=0x7f0aa419df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa419df20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009fd0 tx=0x7f0a98003d00 comp rx=0 tx=0).stop 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 shutdown_connections 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0a8c039140 0x7f0a8c03b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 --2- 172.21.15.94:0/3414115639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0aa4107de0 0x7f0aa419df20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 >> 172.21.15.94:0/3414115639 conn(0x7f0aa4103420 msgr2=0x7f0aa4103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 shutdown_connections 2024-06-03T00:56:43.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:43.135+0000 7f0aabe00700 1 -- 172.21.15.94:0/3414115639 wait complete. 2024-06-03T00:56:45.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.448+0000 7f404e800700 1 Processor -- start 2024-06-03T00:56:45.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.448+0000 7f404e800700 1 -- start start 2024-06-03T00:56:45.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.448+0000 7f404e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404810a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:45.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.448+0000 7f404e800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f404810ab90 con 0x7f404810a1a0 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.449+0000 7f404d400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404810a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.449+0000 7f404d400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404810a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34118/0 (socket says 172.21.15.94:34118) 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.449+0000 7f404d400700 1 -- 172.21.15.94:0/874434783 learned_addr learned my addr 172.21.15.94:0/874434783 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.449+0000 7f404d400700 1 -- 172.21.15.94:0/874434783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404810afd0 con 0x7f404810a1a0 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.450+0000 7f404d400700 1 --2- 172.21.15.94:0/874434783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404810a5c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f403c0098a0 tx=0x7f403c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7f51362d5dc3c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.450+0000 7f4047e00700 1 -- 172.21.15.94:0/874434783 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c004160 con 0x7f404810a1a0 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.450+0000 7f4047e00700 1 -- 172.21.15.94:0/874434783 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403c004300 con 0x7f404810a1a0 2024-06-03T00:56:45.514 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.450+0000 7f4047e00700 1 -- 172.21.15.94:0/874434783 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c011b20 con 0x7f404810a1a0 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.450+0000 7f404e800700 1 -- 172.21.15.94:0/874434783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 msgr2=0x7f404810a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.450+0000 7f404e800700 1 --2- 172.21.15.94:0/874434783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404810a5c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f403c0098a0 tx=0x7f403c009bb0 comp rx=0 tx=0).stop 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.451+0000 7f404e800700 1 -- 172.21.15.94:0/874434783 shutdown_connections 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.451+0000 7f404e800700 1 --2- 172.21.15.94:0/874434783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404810a5c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.451+0000 7f404e800700 1 -- 172.21.15.94:0/874434783 >> 172.21.15.94:0/874434783 conn(0x7f404807d2e0 msgr2=0x7f404807d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.451+0000 7f404e800700 1 -- 172.21.15.94:0/874434783 shutdown_connections 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.451+0000 7f404e800700 1 -- 172.21.15.94:0/874434783 wait complete. 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.452+0000 7f404e800700 1 Processor -- start 2024-06-03T00:56:45.515 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.452+0000 7f404e800700 1 -- start start 2024-06-03T00:56:45.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.452+0000 7f404e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:45.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.452+0000 7f404e800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f404810e380 con 0x7f404810a1a0 2024-06-03T00:56:45.516 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.452+0000 7f404d400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:45.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.452+0000 7f404d400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34134/0 (socket says 172.21.15.94:34134) 2024-06-03T00:56:45.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.452+0000 7f404d400700 1 -- 172.21.15.94:0/791958587 learned_addr learned my addr 172.21.15.94:0/791958587 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:45.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.453+0000 7f404d400700 1 -- 172.21.15.94:0/791958587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404819e2e0 con 0x7f404810a1a0 2024-06-03T00:56:45.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.453+0000 7f404d400700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404819dda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f403c0117e0 tx=0x7f403c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:45.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.453+0000 7f4046000700 1 -- 172.21.15.94:0/791958587 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c003af0 con 0x7f404810a1a0 2024-06-03T00:56:45.517 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.453+0000 7f4046000700 1 -- 172.21.15.94:0/791958587 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403c003c90 con 0x7f404810a1a0 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.453+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404819e570 con 0x7f404810a1a0 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.453+0000 7f4046000700 1 -- 172.21.15.94:0/791958587 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f403c024800 con 0x7f404810a1a0 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.453+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40481a1220 con 0x7f404810a1a0 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.454+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4048063af0 con 0x7f404810a1a0 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.455+0000 7f4046000700 1 -- 172.21.15.94:0/791958587 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f403c022020 con 0x7f404810a1a0 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.455+0000 7f4046000700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40300391a0 0x7f403003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.455+0000 7f4046000700 1 -- 172.21.15.94:0/791958587 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f403c04d4e0 con 0x7f404810a1a0 2024-06-03T00:56:45.518 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.455+0000 7f404ca00700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40300391a0 0x7f403003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.456+0000 7f404ca00700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40300391a0 0x7f403003b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4034006fd0 tx=0x7f4034006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.460+0000 7f4046000700 1 -- 172.21.15.94:0/791958587 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f403c01ac50 con 0x7f404810a1a0 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.792+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f404819e9d0 con 0x7f404810a1a0 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.796+0000 7f4046000700 1 -- 172.21.15.94:0/791958587 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f403c02d3f0 con 0x7f404810a1a0 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.799+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40300391a0 msgr2=0x7f403003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.799+0000 7f404e800700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40300391a0 0x7f403003b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4034006fd0 tx=0x7f4034006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.799+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 msgr2=0x7f404819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.799+0000 7f404e800700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404819dda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f403c0117e0 tx=0x7f403c011ba0 comp rx=0 tx=0).stop 2024-06-03T00:56:45.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.800+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 shutdown_connections 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.800+0000 7f404e800700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40300391a0 0x7f403003b660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.800+0000 7f404e800700 1 --2- 172.21.15.94:0/791958587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f404810a1a0 0x7f404819dda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.800+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 >> 172.21.15.94:0/791958587 conn(0x7f404807d2e0 msgr2=0x7f4048107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.800+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 shutdown_connections 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-06-03T00:56:44.800+0000 7f404e800700 1 -- 172.21.15.94:0/791958587 wait complete. 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-06-03T00:56:45.520 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.521 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-03T00:56:45.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.521 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-03T00:56:45.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.521 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-06-03T00:56:45.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.522 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-06-03T00:56:45.522 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.522 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-03T00:56:45.522 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:56:45.522 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-06-03T00:56:45.579 INFO:tasks.cephadm:Fetching config... 2024-06-03T00:56:45.579 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:56:45.580 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-03T00:56:45.663 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-03T00:56:45.663 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:56:45.663 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-03T00:56:45.747 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-03T00:56:45.747 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:56:45.747 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-03T00:56:45.861 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-03T00:56:45.861 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:56:45.861 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-03T00:56:45.953 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-03T00:56:45.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpqtEOONP72NKuNBgkRlZzASXAWBNE4jgHwCgeq+GLW92oyuuqZ9XqqoLTob41afElXmOidYkkHs3uXlTBuJWBfm9IHQxhHjdQ6FzgFdFUcJSbFGUIMVtVRyK0wMvjH7YaUdbffnjyQL+w1RERWFygXA++jLv/CNnNpl5JyN3aRUX+41Cm834zDxOG4Mn5Fte2xpx+HUevIWmoFVYx+b7idbJwZPqsLzNqB2XlsXDis92xv8s/7raBUiupIkuFkT/C6ZY0uAfwS9jtvOKj9ml28ECKVxPRdmhMefBfJ8J+zMFvFQw9Ow7UWwkmg3gURMswx0sJs8rAVCmGDgcIYKJ7Y2ZDHfdRr90J+2MjEJ/OR3fGAVBrLqu11qMAlugYf2dtFGRSH/AcryDbaU+3wCFht8+cYKItPN6gyBEHWRlBuCFwLKz+PUCBYinwrxah31jEoWNmmsl0FCGGNe87jFPSENnNJNQBE2ZKS5ofhP4C20huxPWMZNk7UC23JW23yBc= ceph-d037ccd0-2143-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T00:56:46.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:45 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/791958587' entity='client.admin' 2024-06-03T00:56:46.098 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpqtEOONP72NKuNBgkRlZzASXAWBNE4jgHwCgeq+GLW92oyuuqZ9XqqoLTob41afElXmOidYkkHs3uXlTBuJWBfm9IHQxhHjdQ6FzgFdFUcJSbFGUIMVtVRyK0wMvjH7YaUdbffnjyQL+w1RERWFygXA++jLv/CNnNpl5JyN3aRUX+41Cm834zDxOG4Mn5Fte2xpx+HUevIWmoFVYx+b7idbJwZPqsLzNqB2XlsXDis92xv8s/7raBUiupIkuFkT/C6ZY0uAfwS9jtvOKj9ml28ECKVxPRdmhMefBfJ8J+zMFvFQw9Ow7UWwkmg3gURMswx0sJs8rAVCmGDgcIYKJ7Y2ZDHfdRr90J+2MjEJ/OR3fGAVBrLqu11qMAlugYf2dtFGRSH/AcryDbaU+3wCFht8+cYKItPN6gyBEHWRlBuCFwLKz+PUCBYinwrxah31jEoWNmmsl0FCGGNe87jFPSENnNJNQBE2ZKS5ofhP4C20huxPWMZNk7UC23JW23yBc= ceph-d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:56:46.132 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpqtEOONP72NKuNBgkRlZzASXAWBNE4jgHwCgeq+GLW92oyuuqZ9XqqoLTob41afElXmOidYkkHs3uXlTBuJWBfm9IHQxhHjdQ6FzgFdFUcJSbFGUIMVtVRyK0wMvjH7YaUdbffnjyQL+w1RERWFygXA++jLv/CNnNpl5JyN3aRUX+41Cm834zDxOG4Mn5Fte2xpx+HUevIWmoFVYx+b7idbJwZPqsLzNqB2XlsXDis92xv8s/7raBUiupIkuFkT/C6ZY0uAfwS9jtvOKj9ml28ECKVxPRdmhMefBfJ8J+zMFvFQw9Ow7UWwkmg3gURMswx0sJs8rAVCmGDgcIYKJ7Y2ZDHfdRr90J+2MjEJ/OR3fGAVBrLqu11qMAlugYf2dtFGRSH/AcryDbaU+3wCFht8+cYKItPN6gyBEHWRlBuCFwLKz+PUCBYinwrxah31jEoWNmmsl0FCGGNe87jFPSENnNJNQBE2ZKS5ofhP4C20huxPWMZNk7UC23JW23yBc= ceph-d037ccd0-2143-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T00:56:46.174 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpqtEOONP72NKuNBgkRlZzASXAWBNE4jgHwCgeq+GLW92oyuuqZ9XqqoLTob41afElXmOidYkkHs3uXlTBuJWBfm9IHQxhHjdQ6FzgFdFUcJSbFGUIMVtVRyK0wMvjH7YaUdbffnjyQL+w1RERWFygXA++jLv/CNnNpl5JyN3aRUX+41Cm834zDxOG4Mn5Fte2xpx+HUevIWmoFVYx+b7idbJwZPqsLzNqB2XlsXDis92xv8s/7raBUiupIkuFkT/C6ZY0uAfwS9jtvOKj9ml28ECKVxPRdmhMefBfJ8J+zMFvFQw9Ow7UWwkmg3gURMswx0sJs8rAVCmGDgcIYKJ7Y2ZDHfdRr90J+2MjEJ/OR3fGAVBrLqu11qMAlugYf2dtFGRSH/AcryDbaU+3wCFht8+cYKItPN6gyBEHWRlBuCFwLKz+PUCBYinwrxah31jEoWNmmsl0FCGGNe87jFPSENnNJNQBE2ZKS5ofhP4C20huxPWMZNk7UC23JW23yBc= ceph-d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:56:46.185 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpqtEOONP72NKuNBgkRlZzASXAWBNE4jgHwCgeq+GLW92oyuuqZ9XqqoLTob41afElXmOidYkkHs3uXlTBuJWBfm9IHQxhHjdQ6FzgFdFUcJSbFGUIMVtVRyK0wMvjH7YaUdbffnjyQL+w1RERWFygXA++jLv/CNnNpl5JyN3aRUX+41Cm834zDxOG4Mn5Fte2xpx+HUevIWmoFVYx+b7idbJwZPqsLzNqB2XlsXDis92xv8s/7raBUiupIkuFkT/C6ZY0uAfwS9jtvOKj9ml28ECKVxPRdmhMefBfJ8J+zMFvFQw9Ow7UWwkmg3gURMswx0sJs8rAVCmGDgcIYKJ7Y2ZDHfdRr90J+2MjEJ/OR3fGAVBrLqu11qMAlugYf2dtFGRSH/AcryDbaU+3wCFht8+cYKItPN6gyBEHWRlBuCFwLKz+PUCBYinwrxah31jEoWNmmsl0FCGGNe87jFPSENnNJNQBE2ZKS5ofhP4C20huxPWMZNk7UC23JW23yBc= ceph-d037ccd0-2143-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-03T00:56:46.224 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpqtEOONP72NKuNBgkRlZzASXAWBNE4jgHwCgeq+GLW92oyuuqZ9XqqoLTob41afElXmOidYkkHs3uXlTBuJWBfm9IHQxhHjdQ6FzgFdFUcJSbFGUIMVtVRyK0wMvjH7YaUdbffnjyQL+w1RERWFygXA++jLv/CNnNpl5JyN3aRUX+41Cm834zDxOG4Mn5Fte2xpx+HUevIWmoFVYx+b7idbJwZPqsLzNqB2XlsXDis92xv8s/7raBUiupIkuFkT/C6ZY0uAfwS9jtvOKj9ml28ECKVxPRdmhMefBfJ8J+zMFvFQw9Ow7UWwkmg3gURMswx0sJs8rAVCmGDgcIYKJ7Y2ZDHfdRr90J+2MjEJ/OR3fGAVBrLqu11qMAlugYf2dtFGRSH/AcryDbaU+3wCFht8+cYKItPN6gyBEHWRlBuCFwLKz+PUCBYinwrxah31jEoWNmmsl0FCGGNe87jFPSENnNJNQBE2ZKS5ofhP4C20huxPWMZNk7UC23JW23yBc= ceph-d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:56:46.237 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-03T00:56:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:56:49.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.814+0000 7f9eb8c00700 1 -- 172.21.15.94:0/595842597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 msgr2=0x7f9eb4104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.814+0000 7f9eb8c00700 1 --2- 172.21.15.94:0/595842597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4104d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009910 tx=0x7f9ea4009c20 comp rx=0 tx=0).stop 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.814+0000 7f9eb8c00700 1 -- 172.21.15.94:0/595842597 shutdown_connections 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.814+0000 7f9eb8c00700 1 --2- 172.21.15.94:0/595842597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4104d80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.814+0000 7f9eb8c00700 1 -- 172.21.15.94:0/595842597 >> 172.21.15.94:0/595842597 conn(0x7f9eb40fc360 msgr2=0x7f9eb40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.814+0000 7f9eb8c00700 1 -- 172.21.15.94:0/595842597 shutdown_connections 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.814+0000 7f9eb8c00700 1 -- 172.21.15.94:0/595842597 wait complete. 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.815+0000 7f9eb8c00700 1 Processor -- start 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.816+0000 7f9eb8c00700 1 -- start start 2024-06-03T00:56:49.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.816+0000 7f9eb8c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.816+0000 7f9eb8c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4107f50 con 0x7f9eb4102990 2024-06-03T00:56:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.816+0000 7f9eb1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.816+0000 7f9eb1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54526/0 (socket says 172.21.15.94:54526) 2024-06-03T00:56:49.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.816+0000 7f9eb1e00700 1 -- 172.21.15.94:0/2728695797 learned_addr learned my addr 172.21.15.94:0/2728695797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:49.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.817+0000 7f9eb1e00700 1 -- 172.21.15.94:0/2728695797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb4102750 con 0x7f9eb4102990 2024-06-03T00:56:49.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.817+0000 7f9eb1e00700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4102210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005950 tx=0x7f9ea4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:49.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.817+0000 7f9eaaa00700 1 -- 172.21.15.94:0/2728695797 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ea4026070 con 0x7f9eb4102990 2024-06-03T00:56:49.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.817+0000 7f9eaaa00700 1 -- 172.21.15.94:0/2728695797 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea400bc30 con 0x7f9eb4102990 2024-06-03T00:56:49.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.817+0000 7f9eaaa00700 1 -- 172.21.15.94:0/2728695797 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ea4003b30 con 0x7f9eb4102990 2024-06-03T00:56:49.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.818+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb4100940 con 0x7f9eb4102990 2024-06-03T00:56:49.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.818+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb4100e80 con 0x7f9eb4102990 2024-06-03T00:56:49.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.819+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb41086d0 con 0x7f9eb4102990 2024-06-03T00:56:49.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.819+0000 7f9eaaa00700 1 -- 172.21.15.94:0/2728695797 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9ea4029070 con 0x7f9eb4102990 2024-06-03T00:56:49.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.820+0000 7f9eaaa00700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e98039150 0x7f9e9803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:49.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.820+0000 7f9eaaa00700 1 -- 172.21.15.94:0/2728695797 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ea401e070 con 0x7f9eb4102990 2024-06-03T00:56:49.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.820+0000 7f9eb1400700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e98039150 0x7f9e9803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:49.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.820+0000 7f9eb1400700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e98039150 0x7f9e9803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006fd0 tx=0x7f9ea0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:49.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:49.825+0000 7f9eaaa00700 1 -- 172.21.15.94:0/2728695797 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ea4013070 con 0x7f9eb4102990 2024-06-03T00:56:50.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.049+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9eb41909d0 con 0x7f9eb4102990 2024-06-03T00:56:50.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.053+0000 7f9eaaa00700 1 -- 172.21.15.94:0/2728695797 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9ea4030020 con 0x7f9eb4102990 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e98039150 msgr2=0x7f9e9803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e98039150 0x7f9e9803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006fd0 tx=0x7f9ea0006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 msgr2=0x7f9eb4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4102210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005950 tx=0x7f9ea4005030 comp rx=0 tx=0).stop 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 shutdown_connections 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e98039150 0x7f9e9803b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 --2- 172.21.15.94:0/2728695797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb4102990 0x7f9eb4102210 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.056+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 >> 172.21.15.94:0/2728695797 conn(0x7f9eb40fc360 msgr2=0x7f9eb40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.057+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 shutdown_connections 2024-06-03T00:56:50.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:50.057+0000 7f9eb8c00700 1 -- 172.21.15.94:0/2728695797 wait complete. 2024-06-03T00:56:50.948 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-03T00:56:50.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-03T00:56:51.283 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:51 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2728695797' entity='client.admin' 2024-06-03T00:56:51.673 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:56:52.836 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:52.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:52.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:56:52.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:52.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:52.837 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:53.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.664+0000 7f0f8e800700 1 -- 172.21.15.94:0/434883155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 msgr2=0x7f0f88104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:53.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.664+0000 7f0f8e800700 1 --2- 172.21.15.94:0/434883155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88104bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009910 tx=0x7f0f7c009c20 comp rx=0 tx=0).stop 2024-06-03T00:56:53.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.664+0000 7f0f8e800700 1 -- 172.21.15.94:0/434883155 shutdown_connections 2024-06-03T00:56:53.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.664+0000 7f0f8e800700 1 --2- 172.21.15.94:0/434883155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88104bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:53.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.664+0000 7f0f8e800700 1 -- 172.21.15.94:0/434883155 >> 172.21.15.94:0/434883155 conn(0x7f0f880fc1f0 msgr2=0x7f0f880fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:53.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.664+0000 7f0f8e800700 1 -- 172.21.15.94:0/434883155 shutdown_connections 2024-06-03T00:56:53.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.664+0000 7f0f8e800700 1 -- 172.21.15.94:0/434883155 wait complete. 2024-06-03T00:56:53.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.665+0000 7f0f8e800700 1 Processor -- start 2024-06-03T00:56:53.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.665+0000 7f0f8e800700 1 -- start start 2024-06-03T00:56:53.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.665+0000 7f0f8e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:53.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.665+0000 7f0f8e800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88107d80 con 0x7f0f881027e0 2024-06-03T00:56:53.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.666+0000 7f0f8d400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:53.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.666+0000 7f0f8d400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54558/0 (socket says 172.21.15.94:54558) 2024-06-03T00:56:53.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.666+0000 7f0f8d400700 1 -- 172.21.15.94:0/3640762722 learned_addr learned my addr 172.21.15.94:0/3640762722 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:53.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.666+0000 7f0f8d400700 1 -- 172.21.15.94:0/3640762722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f881978d0 con 0x7f0f881027e0 2024-06-03T00:56:53.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.667+0000 7f0f8d400700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88197390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005950 tx=0x7f0f7c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:53.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.667+0000 7f0f86000700 1 -- 172.21.15.94:0/3640762722 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f7c026070 con 0x7f0f881027e0 2024-06-03T00:56:53.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.667+0000 7f0f86000700 1 -- 172.21.15.94:0/3640762722 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7c00bbd0 con 0x7f0f881027e0 2024-06-03T00:56:53.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.667+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f88197bc0 con 0x7f0f881027e0 2024-06-03T00:56:53.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.667+0000 7f0f86000700 1 -- 172.21.15.94:0/3640762722 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f7c003ad0 con 0x7f0f881027e0 2024-06-03T00:56:53.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.667+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8819a850 con 0x7f0f881027e0 2024-06-03T00:56:53.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.668+0000 7f0f86000700 1 -- 172.21.15.94:0/3640762722 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0f7c029070 con 0x7f0f881027e0 2024-06-03T00:56:53.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.668+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f88108500 con 0x7f0f881027e0 2024-06-03T00:56:53.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.669+0000 7f0f86000700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f70039150 0x7f0f7003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:53.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.669+0000 7f0f86000700 1 -- 172.21.15.94:0/3640762722 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0f7c01e070 con 0x7f0f881027e0 2024-06-03T00:56:53.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.669+0000 7f0f8ca00700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f70039150 0x7f0f7003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:53.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.670+0000 7f0f8ca00700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f70039150 0x7f0f7003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f74006fd0 tx=0x7f0f74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:53.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.674+0000 7f0f86000700 1 -- 172.21.15.94:0/3640762722 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0f7c013070 con 0x7f0f881027e0 2024-06-03T00:56:53.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.903+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0f8802de90 con 0x7f0f70039150 2024-06-03T00:56:53.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.909+0000 7f0f86000700 1 -- 172.21.15.94:0/3640762722 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f8802de90 con 0x7f0f70039150 2024-06-03T00:56:53.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f70039150 msgr2=0x7f0f7003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:53.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f70039150 0x7f0f7003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f74006fd0 tx=0x7f0f74006e40 comp rx=0 tx=0).stop 2024-06-03T00:56:53.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 msgr2=0x7f0f88197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:53.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88197390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005950 tx=0x7f0f7c005030 comp rx=0 tx=0).stop 2024-06-03T00:56:53.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 shutdown_connections 2024-06-03T00:56:53.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f70039150 0x7f0f7003b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:53.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 --2- 172.21.15.94:0/3640762722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f881027e0 0x7f0f88197390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:53.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 >> 172.21.15.94:0/3640762722 conn(0x7f0f880fc1f0 msgr2=0x7f0f880fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:53.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.912+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 shutdown_connections 2024-06-03T00:56:53.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:53.913+0000 7f0f8e800700 1 -- 172.21.15.94:0/3640762722 wait complete. 2024-06-03T00:56:54.664 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi107 2024-06-03T00:56:54.664 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:56:54.664 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.conf 2024-06-03T00:56:54.682 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:56:54.682 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:56:54.739 INFO:tasks.cephadm:Adding host smithi107 to orchestrator... 2024-06-03T00:56:54.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch host add smithi107 2024-06-03T00:56:55.153 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:56:55.303 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:54 smithi094 ceph-mon[32119]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:55.303 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:55.303 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:56:55.303 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:56:55.303 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:56:55.303 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:54 smithi094 ceph-mon[32119]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-03T00:56:55.303 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:54 smithi094 ceph-mon[32119]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:56:56.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:55 smithi094 ceph-mon[32119]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:56:56.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:55 smithi094 ceph-mon[32119]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:56:57.033 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:56 smithi094 ceph-mon[32119]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T00:56:57.033 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:56 smithi094 ceph-mon[32119]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:56:57.033 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:57.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:57.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:56:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.202+0000 7fe434400700 1 -- 172.21.15.94:0/49899274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 msgr2=0x7fe42c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:56:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.202+0000 7fe434400700 1 --2- 172.21.15.94:0/49899274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c104700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe420009910 tx=0x7fe420009c20 comp rx=0 tx=0).stop 2024-06-03T00:56:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.202+0000 7fe434400700 1 -- 172.21.15.94:0/49899274 shutdown_connections 2024-06-03T00:56:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.202+0000 7fe434400700 1 --2- 172.21.15.94:0/49899274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c104700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:56:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.202+0000 7fe434400700 1 -- 172.21.15.94:0/49899274 >> 172.21.15.94:0/49899274 conn(0x7fe42c0ffbb0 msgr2=0x7fe42c101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:56:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.202+0000 7fe434400700 1 -- 172.21.15.94:0/49899274 shutdown_connections 2024-06-03T00:56:57.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.203+0000 7fe434400700 1 -- 172.21.15.94:0/49899274 wait complete. 2024-06-03T00:56:57.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.203+0000 7fe434400700 1 Processor -- start 2024-06-03T00:56:57.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.204+0000 7fe434400700 1 -- start start 2024-06-03T00:56:57.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.204+0000 7fe434400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:57.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.204+0000 7fe434400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c10e6f0 con 0x7fe42c104320 2024-06-03T00:56:57.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.204+0000 7fe431a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:57.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.204+0000 7fe431a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56192/0 (socket says 172.21.15.94:56192) 2024-06-03T00:56:57.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.204+0000 7fe431a00700 1 -- 172.21.15.94:0/1018844227 learned_addr learned my addr 172.21.15.94:0/1018844227 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:56:57.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.205+0000 7fe431a00700 1 -- 172.21.15.94:0/1018844227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c19fd60 con 0x7fe42c104320 2024-06-03T00:56:57.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.205+0000 7fe431a00700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c19f820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe420006010 tx=0x7fe420005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:57.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.205+0000 7fe42a000700 1 -- 172.21.15.94:0/1018844227 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420026070 con 0x7fe42c104320 2024-06-03T00:56:57.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.205+0000 7fe42a000700 1 -- 172.21.15.94:0/1018844227 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe42000bbd0 con 0x7fe42c104320 2024-06-03T00:56:57.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.205+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c1a0050 con 0x7fe42c104320 2024-06-03T00:56:57.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.206+0000 7fe42a000700 1 -- 172.21.15.94:0/1018844227 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe420003910 con 0x7fe42c104320 2024-06-03T00:56:57.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.206+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c1999e0 con 0x7fe42c104320 2024-06-03T00:56:57.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.207+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42c105800 con 0x7fe42c104320 2024-06-03T00:56:57.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.207+0000 7fe42a000700 1 -- 172.21.15.94:0/1018844227 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe420029070 con 0x7fe42c104320 2024-06-03T00:56:57.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.207+0000 7fe42a000700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe4140390f0 0x7fe41403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:56:57.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.207+0000 7fe42a000700 1 -- 172.21.15.94:0/1018844227 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe42001e070 con 0x7fe42c104320 2024-06-03T00:56:57.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.208+0000 7fe431000700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe4140390f0 0x7fe41403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:56:57.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.208+0000 7fe431000700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe4140390f0 0x7fe41403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe418006fd0 tx=0x7fe418006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:56:57.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.213+0000 7fe42a000700 1 -- 172.21.15.94:0/1018844227 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe420013070 con 0x7fe42c104320 2024-06-03T00:56:57.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:56:57.444+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}) -- 0x7fe42c19af80 con 0x7fe4140390f0 2024-06-03T00:56:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:59 smithi094 ceph-mon[32119]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:56:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:59 smithi094 ceph-mon[32119]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:56:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:56:59 smithi094 ceph-mon[32119]: Deploying cephadm binary to smithi107 2024-06-03T00:57:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:01 smithi094 ceph-mon[32119]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:02.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.103+0000 7fe42a000700 1 -- 172.21.15.94:0/1018844227 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe42c19af80 con 0x7fe4140390f0 2024-06-03T00:57:02.105 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi107' with addr '172.21.15.107' 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe4140390f0 msgr2=0x7fe41403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe4140390f0 0x7fe41403b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe418006fd0 tx=0x7fe418006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 msgr2=0x7fe42c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c19f820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe420006010 tx=0x7fe420005030 comp rx=0 tx=0).stop 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 shutdown_connections 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe4140390f0 0x7fe41403b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 --2- 172.21.15.94:0/1018844227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe42c104320 0x7fe42c19f820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 >> 172.21.15.94:0/1018844227 conn(0x7fe42c0ffbb0 msgr2=0x7fe42c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.106+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 shutdown_connections 2024-06-03T00:57:02.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:02.107+0000 7fe434400700 1 -- 172.21.15.94:0/1018844227 wait complete. 2024-06-03T00:57:02.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-03T00:57:03.300 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:57:03.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:03 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:03.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:03 smithi094 ceph-mon[32119]: Added host smithi107 2024-06-03T00:57:03.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:03 smithi094 ceph-mon[32119]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:03.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:03 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:03.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:03 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:04.496 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:05.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.462+0000 7f7168000700 1 -- 172.21.15.94:0/171730536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 msgr2=0x7f71601046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:05.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.462+0000 7f7168000700 1 --2- 172.21.15.94:0/171730536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f71601046f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7154009910 tx=0x7f7154009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:05.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.463+0000 7f7168000700 1 -- 172.21.15.94:0/171730536 shutdown_connections 2024-06-03T00:57:05.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.463+0000 7f7168000700 1 --2- 172.21.15.94:0/171730536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f71601046f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:05.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.463+0000 7f7168000700 1 -- 172.21.15.94:0/171730536 >> 172.21.15.94:0/171730536 conn(0x7f71600ffbe0 msgr2=0x7f7160102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:05.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.463+0000 7f7168000700 1 -- 172.21.15.94:0/171730536 shutdown_connections 2024-06-03T00:57:05.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.463+0000 7f7168000700 1 -- 172.21.15.94:0/171730536 wait complete. 2024-06-03T00:57:05.464 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:05 smithi094 ceph-mon[32119]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:05.464 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:05 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:05.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.464+0000 7f7168000700 1 Processor -- start 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.464+0000 7f7168000700 1 -- start start 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.464+0000 7f7168000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f716019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.464+0000 7f7168000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f716010e6e0 con 0x7f7160104310 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.465+0000 7f7165600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f716019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.465+0000 7f7165600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f716019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42682/0 (socket says 172.21.15.94:42682) 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.465+0000 7f7165600700 1 -- 172.21.15.94:0/2543818632 learned_addr learned my addr 172.21.15.94:0/2543818632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.465+0000 7f7165600700 1 -- 172.21.15.94:0/2543818632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716019fd50 con 0x7f7160104310 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.465+0000 7f7165600700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f716019f810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7154006010 tx=0x7f7154005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.466+0000 7f715e000700 1 -- 172.21.15.94:0/2543818632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7154026070 con 0x7f7160104310 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.466+0000 7f715e000700 1 -- 172.21.15.94:0/2543818632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715400bbd0 con 0x7f7160104310 2024-06-03T00:57:05.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.466+0000 7f715e000700 1 -- 172.21.15.94:0/2543818632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7154003910 con 0x7f7160104310 2024-06-03T00:57:05.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.466+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71601a0040 con 0x7f7160104310 2024-06-03T00:57:05.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.466+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601999d0 con 0x7f7160104310 2024-06-03T00:57:05.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.467+0000 7f715e000700 1 -- 172.21.15.94:0/2543818632 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7154029070 con 0x7f7160104310 2024-06-03T00:57:05.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.467+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71601057f0 con 0x7f7160104310 2024-06-03T00:57:05.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.467+0000 7f715e000700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7148039150 0x7f714803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:05.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.467+0000 7f715e000700 1 -- 172.21.15.94:0/2543818632 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f715401e070 con 0x7f7160104310 2024-06-03T00:57:05.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.468+0000 7f7164c00700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7148039150 0x7f714803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:05.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.468+0000 7f7164c00700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7148039150 0x7f714803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f714c006fd0 tx=0x7f714c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:05.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.473+0000 7f715e000700 1 -- 172.21.15.94:0/2543818632 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7154013070 con 0x7f7160104310 2024-06-03T00:57:05.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.703+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f716019af00 con 0x7f7148039150 2024-06-03T00:57:05.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.704+0000 7f715e000700 1 -- 172.21.15.94:0/2543818632 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f716019af00 con 0x7f7148039150 2024-06-03T00:57:05.705 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:57:05.705 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}] 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7148039150 msgr2=0x7f714803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7148039150 0x7f714803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f714c006fd0 tx=0x7f714c006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 msgr2=0x7f716019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f716019f810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7154006010 tx=0x7f7154005030 comp rx=0 tx=0).stop 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 shutdown_connections 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7148039150 0x7f714803b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 --2- 172.21.15.94:0/2543818632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7160104310 0x7f716019f810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:05.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.707+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 >> 172.21.15.94:0/2543818632 conn(0x7f71600ffbe0 msgr2=0x7f7160105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:05.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.708+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 shutdown_connections 2024-06-03T00:57:05.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:05.708+0000 7f7168000700 1 -- 172.21.15.94:0/2543818632 wait complete. 2024-06-03T00:57:06.491 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-06-03T00:57:06.491 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:57:06.491 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-06-03T00:57:06.510 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:57:06.510 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:06.567 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-06-03T00:57:06.568 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch host add smithi179 2024-06-03T00:57:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:57:07.401 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:07 smithi094 ceph-mon[32119]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T00:57:07.401 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:07 smithi094 ceph-mon[32119]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:09.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.016+0000 7efe7aa00700 1 -- 172.21.15.94:0/2654769066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 msgr2=0x7efe741024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.016+0000 7efe7aa00700 1 --2- 172.21.15.94:0/2654769066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe741024d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe64009910 tx=0x7efe64009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.016+0000 7efe7aa00700 1 -- 172.21.15.94:0/2654769066 shutdown_connections 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.016+0000 7efe7aa00700 1 --2- 172.21.15.94:0/2654769066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe741024d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.016+0000 7efe7aa00700 1 -- 172.21.15.94:0/2654769066 >> 172.21.15.94:0/2654769066 conn(0x7efe740fd980 msgr2=0x7efe740ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.016+0000 7efe7aa00700 1 -- 172.21.15.94:0/2654769066 shutdown_connections 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.016+0000 7efe7aa00700 1 -- 172.21.15.94:0/2654769066 wait complete. 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.017+0000 7efe7aa00700 1 Processor -- start 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.018+0000 7efe7aa00700 1 -- start start 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.018+0000 7efe7aa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe7419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.018+0000 7efe7aa00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe7410c4c0 con 0x7efe741020f0 2024-06-03T00:57:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.018+0000 7efe73e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe7419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.018+0000 7efe73e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe7419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42686/0 (socket says 172.21.15.94:42686) 2024-06-03T00:57:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.018+0000 7efe73e00700 1 -- 172.21.15.94:0/2565320110 learned_addr learned my addr 172.21.15.94:0/2565320110 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:57:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.019+0000 7efe73e00700 1 -- 172.21.15.94:0/2565320110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7419fd30 con 0x7efe741020f0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.019+0000 7efe73e00700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe7419f7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe64006010 tx=0x7efe64005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.019+0000 7efe70c00700 1 -- 172.21.15.94:0/2565320110 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64026070 con 0x7efe741020f0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.019+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe741a0020 con 0x7efe741020f0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.019+0000 7efe70c00700 1 -- 172.21.15.94:0/2565320110 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6400bbd0 con 0x7efe741020f0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.019+0000 7efe70c00700 1 -- 172.21.15.94:0/2565320110 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe64003910 con 0x7efe741020f0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.019+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741999b0 con 0x7efe741020f0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.020+0000 7efe70c00700 1 -- 172.21.15.94:0/2565320110 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efe64029070 con 0x7efe741020f0 2024-06-03T00:57:09.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.021+0000 7efe70c00700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7efe580390f0 0x7efe5803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.021+0000 7efe70c00700 1 -- 172.21.15.94:0/2565320110 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efe6401e070 con 0x7efe741020f0 2024-06-03T00:57:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.021+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe741035d0 con 0x7efe741020f0 2024-06-03T00:57:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.021+0000 7efe73400700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7efe580390f0 0x7efe5803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.022+0000 7efe73400700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7efe580390f0 0x7efe5803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efe60006fd0 tx=0x7efe60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.027+0000 7efe70c00700 1 -- 172.21.15.94:0/2565320110 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe64030070 con 0x7efe741020f0 2024-06-03T00:57:09.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:09.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:09.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:09.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:09.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:57:09.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:09.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:09.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T00:57:09.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:08 smithi094 ceph-mon[32119]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:09.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:09.258+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) -- 0x7efe7419aee0 con 0x7efe580390f0 2024-06-03T00:57:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:10 smithi094 ceph-mon[32119]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:10 smithi094 ceph-mon[32119]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:10 smithi094 ceph-mon[32119]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T00:57:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:10.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:10.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:11 smithi094 ceph-mon[32119]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:11 smithi094 ceph-mon[32119]: Deploying cephadm binary to smithi179 2024-06-03T00:57:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:11 smithi094 ceph-mon[32119]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:13 smithi094 ceph-mon[32119]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:13.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.860+0000 7efe70c00700 1 -- 172.21.15.94:0/2565320110 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7efe7419aee0 con 0x7efe580390f0 2024-06-03T00:57:13.861 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.864+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7efe580390f0 msgr2=0x7efe5803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.864+0000 7efe7aa00700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7efe580390f0 0x7efe5803b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efe60006fd0 tx=0x7efe60006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.864+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 msgr2=0x7efe7419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.864+0000 7efe7aa00700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe7419f7f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe64006010 tx=0x7efe64005030 comp rx=0 tx=0).stop 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.865+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 shutdown_connections 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.865+0000 7efe7aa00700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7efe580390f0 0x7efe5803b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.865+0000 7efe7aa00700 1 --2- 172.21.15.94:0/2565320110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe741020f0 0x7efe7419f7f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:13.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.865+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 >> 172.21.15.94:0/2565320110 conn(0x7efe740fd980 msgr2=0x7efe74076190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.865+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 shutdown_connections 2024-06-03T00:57:13.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:13.865+0000 7efe7aa00700 1 -- 172.21.15.94:0/2565320110 wait complete. 2024-06-03T00:57:14.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-03T00:57:14.995 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:57:15.150 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:14 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:15.150 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:14 smithi094 ceph-mon[32119]: Added host smithi179 2024-06-03T00:57:15.151 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:14 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:15.151 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:14 smithi094 ceph-mon[32119]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:15.151 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:14 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:16.163 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.954+0000 7fd7cc800700 1 -- 172.21.15.94:0/535373943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 msgr2=0x7fd7c41024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.954+0000 7fd7cc800700 1 --2- 172.21.15.94:0/535373943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c41024e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009910 tx=0x7fd7b8009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.954+0000 7fd7cc800700 1 -- 172.21.15.94:0/535373943 shutdown_connections 2024-06-03T00:57:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.954+0000 7fd7cc800700 1 --2- 172.21.15.94:0/535373943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c41024e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.954+0000 7fd7cc800700 1 -- 172.21.15.94:0/535373943 >> 172.21.15.94:0/535373943 conn(0x7fd7c40fd9d0 msgr2=0x7fd7c40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.954+0000 7fd7cc800700 1 -- 172.21.15.94:0/535373943 shutdown_connections 2024-06-03T00:57:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.955+0000 7fd7cc800700 1 -- 172.21.15.94:0/535373943 wait complete. 2024-06-03T00:57:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.956+0000 7fd7cc800700 1 Processor -- start 2024-06-03T00:57:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.956+0000 7fd7cc800700 1 -- start start 2024-06-03T00:57:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.956+0000 7fd7cc800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.956+0000 7fd7cc800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c410c4d0 con 0x7fd7c4102100 2024-06-03T00:57:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.957+0000 7fd7c9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.957+0000 7fd7c9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44630/0 (socket says 172.21.15.94:44630) 2024-06-03T00:57:16.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.957+0000 7fd7c9e00700 1 -- 172.21.15.94:0/2494749850 learned_addr learned my addr 172.21.15.94:0/2494749850 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:57:16.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.957+0000 7fd7c9e00700 1 -- 172.21.15.94:0/2494749850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c419fd90 con 0x7fd7c4102100 2024-06-03T00:57:16.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.957+0000 7fd7c9e00700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c419f850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8005850 tx=0x7fd7b8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:16.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.958+0000 7fd7bea00700 1 -- 172.21.15.94:0/2494749850 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b8026070 con 0x7fd7c4102100 2024-06-03T00:57:16.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.958+0000 7fd7bea00700 1 -- 172.21.15.94:0/2494749850 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7b800bbd0 con 0x7fd7c4102100 2024-06-03T00:57:16.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.958+0000 7fd7bea00700 1 -- 172.21.15.94:0/2494749850 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b8003970 con 0x7fd7c4102100 2024-06-03T00:57:16.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.958+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c41a0080 con 0x7fd7c4102100 2024-06-03T00:57:16.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.958+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c4199a10 con 0x7fd7c4102100 2024-06-03T00:57:16.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.959+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c41035e0 con 0x7fd7c4102100 2024-06-03T00:57:16.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.959+0000 7fd7bea00700 1 -- 172.21.15.94:0/2494749850 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd7b8029070 con 0x7fd7c4102100 2024-06-03T00:57:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.959+0000 7fd7bea00700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd7ac039150 0x7fd7ac03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.960+0000 7fd7bea00700 1 -- 172.21.15.94:0/2494749850 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd7b801e070 con 0x7fd7c4102100 2024-06-03T00:57:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.960+0000 7fd7c9400700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd7ac039150 0x7fd7ac03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.960+0000 7fd7c9400700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd7ac039150 0x7fd7ac03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006fd0 tx=0x7fd7b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:16.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:16.965+0000 7fd7bea00700 1 -- 172.21.15.94:0/2494749850 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd7b8013070 con 0x7fd7c4102100 2024-06-03T00:57:17.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:16 smithi094 ceph-mon[32119]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:17.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:17.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.193+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7c419af40 con 0x7fd7ac039150 2024-06-03T00:57:17.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.195+0000 7fd7bea00700 1 -- 172.21.15.94:0/2494749850 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fd7c419af40 con 0x7fd7ac039150 2024-06-03T00:57:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T00:57:17.196 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-06-03T00:57:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.198+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd7ac039150 msgr2=0x7fd7ac03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.198+0000 7fd7cc800700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd7ac039150 0x7fd7ac03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006fd0 tx=0x7fd7b4006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.198+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 msgr2=0x7fd7c419f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.198+0000 7fd7cc800700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c419f850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8005850 tx=0x7fd7b8005030 comp rx=0 tx=0).stop 2024-06-03T00:57:17.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.198+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 shutdown_connections 2024-06-03T00:57:17.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.199+0000 7fd7cc800700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd7ac039150 0x7fd7ac03b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:17.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.199+0000 7fd7cc800700 1 --2- 172.21.15.94:0/2494749850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c4102100 0x7fd7c419f850 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:17.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.199+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 >> 172.21.15.94:0/2494749850 conn(0x7fd7c40fd9d0 msgr2=0x7fd7c40746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:17.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.199+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 shutdown_connections 2024-06-03T00:57:17.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:17.199+0000 7fd7cc800700 1 -- 172.21.15.94:0/2494749850 wait complete. 2024-06-03T00:57:17.988 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-03T00:57:17.988 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-03T00:57:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:17 smithi094 ceph-mon[32119]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T00:57:18.413 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:57:19.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:18 smithi094 ceph-mon[32119]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:20.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.410+0000 7fa745c00700 1 -- 172.21.15.94:0/1326601321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 msgr2=0x7fa7401024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:20.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.410+0000 7fa745c00700 1 --2- 172.21.15.94:0/1326601321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa7401024e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa730009910 tx=0x7fa730009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:20.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.410+0000 7fa745c00700 1 -- 172.21.15.94:0/1326601321 shutdown_connections 2024-06-03T00:57:20.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.410+0000 7fa745c00700 1 --2- 172.21.15.94:0/1326601321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa7401024e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:20.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.410+0000 7fa745c00700 1 -- 172.21.15.94:0/1326601321 >> 172.21.15.94:0/1326601321 conn(0x7fa7400fd9d0 msgr2=0x7fa7400ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:20.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.410+0000 7fa745c00700 1 -- 172.21.15.94:0/1326601321 shutdown_connections 2024-06-03T00:57:20.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.410+0000 7fa745c00700 1 -- 172.21.15.94:0/1326601321 wait complete. 2024-06-03T00:57:20.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.411+0000 7fa745c00700 1 Processor -- start 2024-06-03T00:57:20.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.412+0000 7fa745c00700 1 -- start start 2024-06-03T00:57:20.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.412+0000 7fa745c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa74019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:20.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.412+0000 7fa745c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa74010c4d0 con 0x7fa740102100 2024-06-03T00:57:20.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.412+0000 7fa73ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa74019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:20.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.412+0000 7fa73ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa74019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44644/0 (socket says 172.21.15.94:44644) 2024-06-03T00:57:20.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.412+0000 7fa73ea00700 1 -- 172.21.15.94:0/2733033985 learned_addr learned my addr 172.21.15.94:0/2733033985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:57:20.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.413+0000 7fa73ea00700 1 -- 172.21.15.94:0/2733033985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74019dbc0 con 0x7fa740102100 2024-06-03T00:57:20.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.413+0000 7fa73ea00700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa74019d680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa730006010 tx=0x7fa730005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:20.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.413+0000 7fa737400700 1 -- 172.21.15.94:0/2733033985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa730026070 con 0x7fa740102100 2024-06-03T00:57:20.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.413+0000 7fa737400700 1 -- 172.21.15.94:0/2733033985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73000bbd0 con 0x7fa740102100 2024-06-03T00:57:20.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.413+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74019deb0 con 0x7fa740102100 2024-06-03T00:57:20.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.413+0000 7fa737400700 1 -- 172.21.15.94:0/2733033985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa730003910 con 0x7fa740102100 2024-06-03T00:57:20.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.414+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa740197800 con 0x7fa740102100 2024-06-03T00:57:20.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.415+0000 7fa737400700 1 -- 172.21.15.94:0/2733033985 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa730029070 con 0x7fa740102100 2024-06-03T00:57:20.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.415+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7401035e0 con 0x7fa740102100 2024-06-03T00:57:20.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.415+0000 7fa737400700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa724039150 0x7fa72403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:20.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.415+0000 7fa737400700 1 -- 172.21.15.94:0/2733033985 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa73001e070 con 0x7fa740102100 2024-06-03T00:57:20.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.416+0000 7fa73e000700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa724039150 0x7fa72403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:20.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.416+0000 7fa73e000700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa724039150 0x7fa72403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa72c006fd0 tx=0x7fa72c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:20.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.421+0000 7fa737400700 1 -- 172.21.15.94:0/2733033985 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa730013070 con 0x7fa740102100 2024-06-03T00:57:20.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.652+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa740198dd0 con 0x7fa740102100 2024-06-03T00:57:20.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.831+0000 7fa737400700 1 -- 172.21.15.94:0/2733033985 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa730030020 con 0x7fa740102100 2024-06-03T00:57:20.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa724039150 msgr2=0x7fa72403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa724039150 0x7fa72403b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa72c006fd0 tx=0x7fa72c006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 msgr2=0x7fa74019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa74019d680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa730006010 tx=0x7fa730005030 comp rx=0 tx=0).stop 2024-06-03T00:57:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 shutdown_connections 2024-06-03T00:57:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa724039150 0x7fa72403b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 --2- 172.21.15.94:0/2733033985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740102100 0x7fa74019d680 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:20.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.835+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 >> 172.21.15.94:0/2733033985 conn(0x7fa7400fd9d0 msgr2=0x7fa740103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:20.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.836+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 shutdown_connections 2024-06-03T00:57:20.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:20.836+0000 7fa745c00700 1 -- 172.21.15.94:0/2733033985 wait complete. 2024-06-03T00:57:20.837 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-06-03T00:57:21.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:21.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:21.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:21.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:21.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:57:21.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:21.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:21.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T00:57:21.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:21.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:21.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:20 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2733033985' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T00:57:21.625 INFO:tasks.cephadm:Adding mon.a on smithi094 2024-06-03T00:57:21.625 INFO:tasks.cephadm:Adding mon.b on smithi107 2024-06-03T00:57:21.626 INFO:tasks.cephadm:Adding mon.c on smithi179 2024-06-03T00:57:21.626 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c' 2024-06-03T00:57:21.853 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:22.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:21 smithi094 ceph-mon[32119]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:22.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:21 smithi094 ceph-mon[32119]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T00:57:22.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:21 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2733033985' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T00:57:22.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:21 smithi094 ceph-mon[32119]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:22.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:22.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:22.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:23.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:22 smithi094 ceph-mon[32119]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:23.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.389+0000 7f0c51c73700 1 -- 172.21.15.179:0/328874784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 msgr2=0x7f0c4c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:23.391 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.389+0000 7f0c51c73700 1 --2- 172.21.15.179:0/328874784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c1024e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009910 tx=0x7f0c3c009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.389+0000 7f0c51c73700 1 -- 172.21.15.179:0/328874784 shutdown_connections 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.389+0000 7f0c51c73700 1 --2- 172.21.15.179:0/328874784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c1024e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.389+0000 7f0c51c73700 1 -- 172.21.15.179:0/328874784 >> 172.21.15.179:0/328874784 conn(0x7f0c4c0fd9d0 msgr2=0x7f0c4c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.389+0000 7f0c51c73700 1 -- 172.21.15.179:0/328874784 shutdown_connections 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.389+0000 7f0c51c73700 1 -- 172.21.15.179:0/328874784 wait complete. 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.390+0000 7f0c51c73700 1 Processor -- start 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.390+0000 7f0c51c73700 1 -- start start 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.390+0000 7f0c51c73700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.390+0000 7f0c51c73700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10c4d0 con 0x7f0c4c102100 2024-06-03T00:57:23.392 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.391+0000 7f0c4b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:23.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.391+0000 7f0c4b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.179:56362/0 (socket says 172.21.15.179:56362) 2024-06-03T00:57:23.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.391+0000 7f0c4b7fe700 1 -- 172.21.15.179:0/2895055678 learned_addr learned my addr 172.21.15.179:0/2895055678 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T00:57:23.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.392+0000 7f0c4b7fe700 1 -- 172.21.15.179:0/2895055678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c19dbc0 con 0x7f0c4c102100 2024-06-03T00:57:23.393 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.392+0000 7f0c4b7fe700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c19d680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c005850 tx=0x7f0c3c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:23.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.393+0000 7f0c48ff9700 1 -- 172.21.15.179:0/2895055678 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c026070 con 0x7f0c4c102100 2024-06-03T00:57:23.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.393+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c19deb0 con 0x7f0c4c102100 2024-06-03T00:57:23.394 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.393+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c197800 con 0x7f0c4c102100 2024-06-03T00:57:23.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.393+0000 7f0c48ff9700 1 -- 172.21.15.179:0/2895055678 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c00bbd0 con 0x7f0c4c102100 2024-06-03T00:57:23.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.393+0000 7f0c48ff9700 1 -- 172.21.15.179:0/2895055678 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c0039b0 con 0x7f0c4c102100 2024-06-03T00:57:23.395 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.394+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4c1035e0 con 0x7f0c4c102100 2024-06-03T00:57:23.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.394+0000 7f0c48ff9700 1 -- 172.21.15.179:0/2895055678 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0c3c029070 con 0x7f0c4c102100 2024-06-03T00:57:23.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.394+0000 7f0c48ff9700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0c34038d00 0x7f0c3403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:23.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.394+0000 7f0c48ff9700 1 -- 172.21.15.179:0/2895055678 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0c3c01e070 con 0x7f0c4c102100 2024-06-03T00:57:23.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.395+0000 7f0c4affd700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0c34038d00 0x7f0c3403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:23.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.396+0000 7f0c4affd700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0c34038d00 0x7f0c3403b1c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006fd0 tx=0x7f0c40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:23.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.411+0000 7f0c48ff9700 1 -- 172.21.15.179:0/2895055678 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0c3c013070 con 0x7f0c4c102100 2024-06-03T00:57:23.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.617+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}) -- 0x7f0c4c198d30 con 0x7f0c34038d00 2024-06-03T00:57:23.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.632+0000 7f0c48ff9700 1 -- 172.21.15.179:0/2895055678 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0c4c198d30 con 0x7f0c34038d00 2024-06-03T00:57:23.632 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-06-03T00:57:23.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0c34038d00 msgr2=0x7f0c3403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:23.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0c34038d00 0x7f0c3403b1c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006fd0 tx=0x7f0c40006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:23.634 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 msgr2=0x7f0c4c19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:23.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c19d680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c005850 tx=0x7f0c3c005030 comp rx=0 tx=0).stop 2024-06-03T00:57:23.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 shutdown_connections 2024-06-03T00:57:23.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0c34038d00 0x7f0c3403b1c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:23.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 --2- 172.21.15.179:0/2895055678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c4c102100 0x7f0c4c19d680 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:23.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 >> 172.21.15.179:0/2895055678 conn(0x7f0c4c0fd9d0 msgr2=0x7f0c4c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:23.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 shutdown_connections 2024-06-03T00:57:23.635 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:23.633+0000 7f0c51c73700 1 -- 172.21.15.179:0/2895055678 wait complete. 2024-06-03T00:57:24.276 DEBUG:teuthology.orchestra.run.smithi107:mon.b> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.b.service 2024-06-03T00:57:24.279 DEBUG:teuthology.orchestra.run.smithi179:mon.c> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.c.service 2024-06-03T00:57:24.281 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T00:57:24.282 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T00:57:25.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c;count:3 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: Deploying daemon mon.c on smithi179 2024-06-03T00:57:25.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:24 smithi094 ceph-mon[32119]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:25.138 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.935+0000 7fda61045700 1 -- 172.21.15.179:0/2308651725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 msgr2=0x7fda5c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.935+0000 7fda61045700 1 --2- 172.21.15.179:0/2308651725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c101420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009910 tx=0x7fda4c009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.935+0000 7fda61045700 1 -- 172.21.15.179:0/2308651725 shutdown_connections 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.935+0000 7fda61045700 1 --2- 172.21.15.179:0/2308651725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c101420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.935+0000 7fda61045700 1 -- 172.21.15.179:0/2308651725 >> 172.21.15.179:0/2308651725 conn(0x7fda5c0768f0 msgr2=0x7fda5c076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.935+0000 7fda61045700 1 -- 172.21.15.179:0/2308651725 shutdown_connections 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.935+0000 7fda61045700 1 -- 172.21.15.179:0/2308651725 wait complete. 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.936+0000 7fda61045700 1 Processor -- start 2024-06-03T00:57:26.937 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.936+0000 7fda61045700 1 -- start start 2024-06-03T00:57:26.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.936+0000 7fda61045700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:26.938 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.936+0000 7fda61045700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10b3c0 con 0x7fda5c101040 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.937+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.937+0000 7fda5ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.179:56386/0 (socket says 172.21.15.179:56386) 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.937+0000 7fda5ad9d700 1 -- 172.21.15.179:0/2246749047 learned_addr learned my addr 172.21.15.179:0/2246749047 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.937+0000 7fda5ad9d700 1 -- 172.21.15.179:0/2246749047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c19db90 con 0x7fda5c101040 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.938+0000 7fda5ad9d700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c19d650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda4c005850 tx=0x7fda4c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.938+0000 7fda43fff700 1 -- 172.21.15.179:0/2246749047 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c026070 con 0x7fda5c101040 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.938+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c19de80 con 0x7fda5c101040 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.938+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c1977d0 con 0x7fda5c101040 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.938+0000 7fda43fff700 1 -- 172.21.15.179:0/2246749047 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c00bbd0 con 0x7fda5c101040 2024-06-03T00:57:26.939 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.938+0000 7fda43fff700 1 -- 172.21.15.179:0/2246749047 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c0047e0 con 0x7fda5c101040 2024-06-03T00:57:26.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.939+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda48005330 con 0x7fda5c101040 2024-06-03T00:57:26.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.940+0000 7fda43fff700 1 -- 172.21.15.179:0/2246749047 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fda4c029070 con 0x7fda5c101040 2024-06-03T00:57:26.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.940+0000 7fda43fff700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda440390a0 0x7fda4403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:26.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.940+0000 7fda43fff700 1 -- 172.21.15.179:0/2246749047 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fda4c01e070 con 0x7fda5c101040 2024-06-03T00:57:26.941 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.940+0000 7fda5a59c700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda440390a0 0x7fda4403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:26.942 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.942+0000 7fda5a59c700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda440390a0 0x7fda4403b560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda50006fd0 tx=0x7fda50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:26.955 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:26.955+0000 7fda43fff700 1 -- 172.21.15.179:0/2246749047 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda4c013070 con 0x7fda5c101040 2024-06-03T00:57:27.240 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.240+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fda48005620 con 0x7fda5c101040 2024-06-03T00:57:27.241 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.241+0000 7fda43fff700 1 -- 172.21.15.179:0/2246749047 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fda4c029560 con 0x7fda5c101040 2024-06-03T00:57:27.241 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:57:27.242 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","modified":"2024-06-03T00:55:21.644119Z","created":"2024-06-03T00:55:21.644119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T00:57:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda440390a0 msgr2=0x7fda4403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:27.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda440390a0 0x7fda4403b560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda50006fd0 tx=0x7fda50006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 msgr2=0x7fda5c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c19d650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda4c005850 tx=0x7fda4c005030 comp rx=0 tx=0).stop 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 shutdown_connections 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda440390a0 0x7fda4403b560 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 --2- 172.21.15.179:0/2246749047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda5c101040 0x7fda5c19d650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 >> 172.21.15.179:0/2246749047 conn(0x7fda5c0768f0 msgr2=0x7fda5c0698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 shutdown_connections 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:27.243+0000 7fda61045700 1 -- 172.21.15.179:0/2246749047 wait complete. 2024-06-03T00:57:27.244 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-06-03T00:57:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:27 smithi094 ceph-mon[32119]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:28 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/2246749047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:28.844 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T00:57:28.845 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T00:57:29.199 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T00:57:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:29 smithi094 ceph-mon[32119]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:30.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.675+0000 7fea3e555700 1 -- 172.21.15.179:0/2041432477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 msgr2=0x7fea38103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:30.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.675+0000 7fea3e555700 1 --2- 172.21.15.179:0/2041432477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38103540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea28009910 tx=0x7fea28009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.675+0000 7fea3e555700 1 -- 172.21.15.179:0/2041432477 shutdown_connections 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.675+0000 7fea3e555700 1 --2- 172.21.15.179:0/2041432477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38103540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.675+0000 7fea3e555700 1 -- 172.21.15.179:0/2041432477 >> 172.21.15.179:0/2041432477 conn(0x7fea380fc9d0 msgr2=0x7fea380fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.675+0000 7fea3e555700 1 -- 172.21.15.179:0/2041432477 shutdown_connections 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.675+0000 7fea3e555700 1 -- 172.21.15.179:0/2041432477 wait complete. 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.676+0000 7fea3e555700 1 Processor -- start 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.676+0000 7fea3e555700 1 -- start start 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.676+0000 7fea3e555700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.676+0000 7fea3e555700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea3810c650 con 0x7fea38103160 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.677+0000 7fea37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:30.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.677+0000 7fea37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.179:38322/0 (socket says 172.21.15.179:38322) 2024-06-03T00:57:30.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.677+0000 7fea37fff700 1 -- 172.21.15.179:0/2296169419 learned_addr learned my addr 172.21.15.179:0/2296169419 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.677+0000 7fea37fff700 1 -- 172.21.15.179:0/2296169419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3819a2a0 con 0x7fea38103160 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.678+0000 7fea37fff700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38199d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea28005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.678+0000 7fea357fa700 1 -- 172.21.15.179:0/2296169419 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28026070 con 0x7fea38103160 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.678+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3819a530 con 0x7fea38103160 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.678+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3819a990 con 0x7fea38103160 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.678+0000 7fea357fa700 1 -- 172.21.15.179:0/2296169419 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2800bc30 con 0x7fea38103160 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.678+0000 7fea357fa700 1 -- 172.21.15.179:0/2296169419 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28003a50 con 0x7fea38103160 2024-06-03T00:57:30.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.679+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea24005330 con 0x7fea38103160 2024-06-03T00:57:30.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.680+0000 7fea357fa700 1 -- 172.21.15.179:0/2296169419 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fea28029070 con 0x7fea38103160 2024-06-03T00:57:30.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.680+0000 7fea357fa700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea200390f0 0x7fea2003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:30.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.680+0000 7fea357fa700 1 -- 172.21.15.179:0/2296169419 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fea2801e070 con 0x7fea38103160 2024-06-03T00:57:30.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.680+0000 7fea377fe700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea200390f0 0x7fea2003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:30.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.681+0000 7fea377fe700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea200390f0 0x7fea2003b5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006fd0 tx=0x7fea2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:30.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.695+0000 7fea357fa700 1 -- 172.21.15.179:0/2296169419 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea28013070 con 0x7fea38103160 2024-06-03T00:57:30.983 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.982+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fea240058f0 con 0x7fea38103160 2024-06-03T00:57:30.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.983+0000 7fea357fa700 1 -- 172.21.15.179:0/2296169419 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fea28034090 con 0x7fea38103160 2024-06-03T00:57:30.984 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:57:30.984 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","modified":"2024-06-03T00:55:21.644119Z","created":"2024-06-03T00:55:21.644119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-03T00:57:30.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea200390f0 msgr2=0x7fea2003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:30.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea200390f0 0x7fea2003b5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006fd0 tx=0x7fea2c006e40 comp rx=0 tx=0).stop 2024-06-03T00:57:30.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 msgr2=0x7fea38199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:30.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38199d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea28006010 tx=0x7fea28005050 comp rx=0 tx=0).stop 2024-06-03T00:57:30.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 shutdown_connections 2024-06-03T00:57:30.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea200390f0 0x7fea2003b5b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:30.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 --2- 172.21.15.179:0/2296169419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea38103160 0x7fea38199d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:30.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 >> 172.21.15.179:0/2296169419 conn(0x7fea380fc9d0 msgr2=0x7fea38108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:30.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 shutdown_connections 2024-06-03T00:57:30.988 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:30.986+0000 7fea3e555700 1 -- 172.21.15.179:0/2296169419 wait complete. 2024-06-03T00:57:30.988 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-06-03T00:57:31.027 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-03T00:57:31.028 INFO:journalctl@ceph.mon.c.smithi179.stdout: e1 2024-06-03T00:57:31.028 INFO:journalctl@ceph.mon.c.smithi179.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T00:57:31.028 INFO:journalctl@ceph.mon.c.smithi179.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T00:57:31.028 INFO:journalctl@ceph.mon.c.smithi179.stdout: legacy client fscid: -1 2024-06-03T00:57:31.028 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-06-03T00:57:31.028 INFO:journalctl@ceph.mon.c.smithi179.stdout: No filesystems configured 2024-06-03T00:57:31.028 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-03T00:57:31.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-03T00:57:31.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Deploying cephadm binary to smithi107 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Added host smithi107 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:57:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:31.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:31.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T00:57:31.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:31.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:31.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T00:57:31.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Deploying cephadm binary to smithi179 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Added host smithi179 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T00:57:31.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2733033985' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2733033985' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c;count:3 2024-06-03T00:57:31.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: Deploying daemon mon.c on smithi179 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/2246749047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 ceph-mon[37041]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-03T00:57:31.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 bash[36999]: a77623c6a7daa68cc507f22ae1abe2a6ae236f42bcccb6690812540964964db7 2024-06-03T00:57:31.363 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:31 smithi179 systemd[1]: Started Ceph mon.c for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:57:32.642 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T00:57:32.642 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T00:57:32.885 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T00:57:36.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:57:36.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: mon.a calling monitor election 2024-06-03T00:57:36.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:36.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: mon.c calling monitor election 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: monmap epoch 2 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: last_changed 2024-06-03T00:57:31.035187+0000 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: min_mon_release 19 (squid) 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: election_strategy: 1 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:36.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: fsmap 2024-06-03T00:57:36.603 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:36.603 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: mgrmap e13: x(active, since 61s) 2024-06-03T00:57:36.603 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: overall HEALTH_OK 2024-06-03T00:57:36.603 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:36.603 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:57:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: mon.a calling monitor election 2024-06-03T00:57:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: mon.c calling monitor election 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: monmap epoch 2 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: last_changed 2024-06-03T00:57:31.035187+0000 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: min_mon_release 19 (squid) 2024-06-03T00:57:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: election_strategy: 1 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: fsmap 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: mgrmap e13: x(active, since 61s) 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: overall HEALTH_OK 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:36.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:37.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.241+0000 7fc436604700 1 -- 172.21.15.179:0/1086798710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 msgr2=0x7fc418005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.242+0000 7fc436604700 1 --2- 172.21.15.179:0/1086798710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 0x7fc418005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc430101ee0 tx=0x7fc42000feb0 comp rx=0 tx=0).stop 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.242+0000 7fc436604700 1 -- 172.21.15.179:0/1086798710 shutdown_connections 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.242+0000 7fc436604700 1 --2- 172.21.15.179:0/1086798710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 0x7fc418005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.242+0000 7fc436604700 1 -- 172.21.15.179:0/1086798710 >> 172.21.15.179:0/1086798710 conn(0x7fc4300fdaa0 msgr2=0x7fc4300ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.242+0000 7fc436604700 1 -- 172.21.15.179:0/1086798710 shutdown_connections 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.242+0000 7fc436604700 1 -- 172.21.15.179:0/1086798710 wait complete. 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.242+0000 7fc436604700 1 Processor -- start 2024-06-03T00:57:37.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc436604700 1 -- start start 2024-06-03T00:57:37.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc436604700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 0x7fc43019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:37.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc436604700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 0x7fc4301979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:37.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc436604700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc43010a040 con 0x7fc4180035c0 2024-06-03T00:57:37.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc436604700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc430109ec0 con 0x7fc43019dd30 2024-06-03T00:57:37.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc42f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 0x7fc4301979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:37.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc42f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 0x7fc4301979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46076/0 (socket says 172.21.15.179:46076) 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.243+0000 7fc42f7fe700 1 -- 172.21.15.179:0/3500932177 learned_addr learned my addr 172.21.15.179:0/3500932177 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42f7fe700 1 -- 172.21.15.179:0/3500932177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 msgr2=0x7fc4301979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42f7fe700 1 -- 172.21.15.179:0/3500932177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 msgr2=0x7fc4301979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42f7fe700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 0x7fc4301979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42f7fe700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 0x7fc4301979b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42ffff700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 0x7fc43019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42ffff700 1 -- 172.21.15.179:0/3500932177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 msgr2=0x7fc4301979b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42ffff700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 0x7fc4301979b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:37.246 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.244+0000 7fc42ffff700 1 -- 172.21.15.179:0/3500932177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43019e370 con 0x7fc4180035c0 2024-06-03T00:57:37.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.245+0000 7fc42ffff700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 0x7fc43019d7f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4200059d0 tx=0x7fc4200057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:37.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.245+0000 7fc42d7fa700 1 -- 172.21.15.179:0/3500932177 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc420004b20 con 0x7fc4180035c0 2024-06-03T00:57:37.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.245+0000 7fc42d7fa700 1 -- 172.21.15.179:0/3500932177 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc420013e40 con 0x7fc4180035c0 2024-06-03T00:57:37.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.245+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc430197ef0 con 0x7fc4180035c0 2024-06-03T00:57:37.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.245+0000 7fc42d7fa700 1 -- 172.21.15.179:0/3500932177 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc420011cd0 con 0x7fc4180035c0 2024-06-03T00:57:37.247 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.245+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc430198480 con 0x7fc4180035c0 2024-06-03T00:57:37.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.247+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4301036f0 con 0x7fc4180035c0 2024-06-03T00:57:37.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.247+0000 7fc42d7fa700 1 -- 172.21.15.179:0/3500932177 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc420023070 con 0x7fc4180035c0 2024-06-03T00:57:37.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.247+0000 7fc42d7fa700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc41c038e90 0x7fc41c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:37.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.247+0000 7fc42d7fa700 1 -- 172.21.15.179:0/3500932177 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc420020050 con 0x7fc4180035c0 2024-06-03T00:57:37.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.248+0000 7fc42f7fe700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc41c038e90 0x7fc41c03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:37.249 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.249+0000 7fc42f7fe700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc41c038e90 0x7fc41c03b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc424009180 tx=0x7fc424008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:37.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.262+0000 7fc42d7fa700 1 -- 172.21.15.179:0/3500932177 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc420031020 con 0x7fc4180035c0 2024-06-03T00:57:37.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.540+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc430198ee0 con 0x7fc4180035c0 2024-06-03T00:57:37.541 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:37.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:37.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:37.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:37 smithi179 ceph-mon[37041]: Deploying daemon mon.b on smithi107 2024-06-03T00:57:37.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:37 smithi179 ceph-mon[37041]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:37.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:37.542 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.542+0000 7fc42d7fa700 1 -- 172.21.15.179:0/3500932177 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc420028030 con 0x7fc4180035c0 2024-06-03T00:57:37.543 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:57:37.543 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","modified":"2024-06-03T00:57:31.035187Z","created":"2024-06-03T00:55:21.644119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc41c038e90 msgr2=0x7fc41c03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc41c038e90 0x7fc41c03b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc424009180 tx=0x7fc424008040 comp rx=0 tx=0).stop 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 msgr2=0x7fc43019d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 0x7fc43019d7f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc4200059d0 tx=0x7fc4200057d0 comp rx=0 tx=0).stop 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 shutdown_connections 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc41c038e90 0x7fc41c03b350 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4180035c0 0x7fc43019d7f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 --2- 172.21.15.179:0/3500932177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc43019dd30 0x7fc4301979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 >> 172.21.15.179:0/3500932177 conn(0x7fc4300fdaa0 msgr2=0x7fc430100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 shutdown_connections 2024-06-03T00:57:37.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:37.544+0000 7fc436604700 1 -- 172.21.15.179:0/3500932177 wait complete. 2024-06-03T00:57:37.546 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-06-03T00:57:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:37 smithi094 ceph-mon[32119]: Deploying daemon mon.b on smithi107 2024-06-03T00:57:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:37 smithi094 ceph-mon[32119]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:38.287 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:38 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/3500932177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:38.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:38 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/3500932177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:39.137 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-03T00:57:39.138 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-03T00:57:39.378 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T00:57:39.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:39 smithi179 ceph-mon[37041]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:39.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:39 smithi094 ceph-mon[32119]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).mds e1 new map 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).mds e1 print_map 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout: e1 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout: legacy client fscid: -1 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout: 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout: No filesystems configured 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-03T00:57:39.894 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-03T00:57:39.895 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-03T00:57:39.895 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-03T00:57:39.895 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-03T00:57:39.895 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Deploying cephadm binary to smithi107 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Added host smithi107 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Deploying cephadm binary to smithi179 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Added host smithi179 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.287 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2733033985' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-03T00:57:40.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2733033985' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c;count:3 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Deploying daemon mon.c on smithi179 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/2246749047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.a calling monitor election 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.c calling monitor election 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:40.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: monmap epoch 2 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: last_changed 2024-06-03T00:57:31.035187+0000 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: min_mon_release 19 (squid) 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: election_strategy: 1 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: fsmap 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mgrmap e13: x(active, since 61s) 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: overall HEALTH_OK 2024-06-03T00:57:40.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: Deploying daemon mon.b on smithi107 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/3500932177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 ceph-mon[36622]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 bash[36602]: 62de82cd24ed69ae7a4739efee425b61d72fb84745fcc4340bb62eaf5ce34379 2024-06-03T00:57:40.293 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:39 smithi107 systemd[1]: Started Ceph mon.b for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:57:45.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.174+0000 7f027e5d7700 1 -- 172.21.15.179:0/684891092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 msgr2=0x7f0260005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:45.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.174+0000 7f027e5d7700 1 --2- 172.21.15.179:0/684891092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 0x7f0260005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0278074700 tx=0x7f026800fe60 comp rx=0 tx=0).stop 2024-06-03T00:57:45.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.174+0000 7f027e5d7700 1 -- 172.21.15.179:0/684891092 shutdown_connections 2024-06-03T00:57:45.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.174+0000 7f027e5d7700 1 --2- 172.21.15.179:0/684891092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 0x7f0260005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.174+0000 7f027e5d7700 1 -- 172.21.15.179:0/684891092 >> 172.21.15.179:0/684891092 conn(0x7f02780fdaa0 msgr2=0x7f02780ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:45.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.174+0000 7f027e5d7700 1 -- 172.21.15.179:0/684891092 shutdown_connections 2024-06-03T00:57:45.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.174+0000 7f027e5d7700 1 -- 172.21.15.179:0/684891092 wait complete. 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.175+0000 7f027e5d7700 1 Processor -- start 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.175+0000 7f027e5d7700 1 -- start start 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027e5d7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 0x7f0278199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027e5d7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 0x7f027819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027e5d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 0x7f027819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027e5d7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f027810c220 con 0x7f02600035c0 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027e5d7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f027810c0a0 con 0x7f027819ec80 2024-06-03T00:57:45.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027e5d7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f027810c3a0 con 0x7f027819a3b0 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f02777fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 0x7f027819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f02777fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 0x7f027819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46372/0 (socket says 172.21.15.179:46372) 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f02777fe700 1 -- 172.21.15.179:0/1038443261 learned_addr learned my addr 172.21.15.179:0/1038443261 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027cb74700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 0x7f027819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f02777fe700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 msgr2=0x7f027819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f02777fe700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 msgr2=0x7f027819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f02777fe700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 0x7f027819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f0277fff700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 0x7f0278199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f02777fe700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 0x7f027819e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027cb74700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 msgr2=0x7f027819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027cb74700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 msgr2=0x7f027819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027cb74700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 0x7f027819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-03T00:57:45.179 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.176+0000 7f027cb74700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 0x7f027819f130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T00:57:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.177+0000 7f0277fff700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 msgr2=0x7f027819e740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T00:57:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.177+0000 7f0277fff700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 0x7f027819e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.177+0000 7f0277fff700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 msgr2=0x7f027819f130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T00:57:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.177+0000 7f0277fff700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 0x7f027819f130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.177+0000 7f0277fff700 1 -- 172.21.15.179:0/1038443261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02781a69f0 con 0x7f02600035c0 2024-06-03T00:57:45.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.177+0000 7f0277fff700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 0x7f0278199e70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0278074700 tx=0x7f0268014fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:45.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.178+0000 7f02757fa700 1 -- 172.21.15.179:0/1038443261 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268022040 con 0x7f02600035c0 2024-06-03T00:57:45.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.178+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02781a6bc0 con 0x7f02600035c0 2024-06-03T00:57:45.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.178+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a7150 con 0x7f02600035c0 2024-06-03T00:57:45.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.178+0000 7f02757fa700 1 -- 172.21.15.179:0/1038443261 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f026802c070 con 0x7f02600035c0 2024-06-03T00:57:45.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.178+0000 7f02757fa700 1 -- 172.21.15.179:0/1038443261 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268004170 con 0x7f02600035c0 2024-06-03T00:57:45.181 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.179+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0278063af0 con 0x7f02600035c0 2024-06-03T00:57:45.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.180+0000 7f02757fa700 1 -- 172.21.15.179:0/1038443261 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0268012c70 con 0x7f02600035c0 2024-06-03T00:57:45.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.180+0000 7f02757fa700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0264039580 0x7f026403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:45.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.180+0000 7f02757fa700 1 -- 172.21.15.179:0/1038443261 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f026804e6b0 con 0x7f02600035c0 2024-06-03T00:57:45.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.180+0000 7f02777fe700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0264039580 0x7f026403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:45.182 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.181+0000 7f02777fe700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0264039580 0x7f026403ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f026c009180 tx=0x7f026c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:45.195 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.195+0000 7f02757fa700 1 -- 172.21.15.179:0/1038443261 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0268037020 con 0x7f02600035c0 2024-06-03T00:57:45.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: mon.c calling monitor election 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: mon.a calling monitor election 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: monmap epoch 3 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: last_changed 2024-06-03T00:57:39.899935+0000 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: min_mon_release 19 (squid) 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: election_strategy: 1 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-06-03T00:57:45.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: fsmap 2024-06-03T00:57:45.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:45.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: mgrmap e13: x(active, since 70s) 2024-06-03T00:57:45.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: overall HEALTH_OK 2024-06-03T00:57:45.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:45.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: mon.c calling monitor election 2024-06-03T00:57:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:57:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: mon.a calling monitor election 2024-06-03T00:57:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: monmap epoch 3 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: last_changed 2024-06-03T00:57:39.899935+0000 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: min_mon_release 19 (squid) 2024-06-03T00:57:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: election_strategy: 1 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: fsmap 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: mgrmap e13: x(active, since 70s) 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: overall HEALTH_OK 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:45.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:45.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.490+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f027802dca0 con 0x7f02600035c0 2024-06-03T00:57:45.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.491+0000 7f02757fa700 1 -- 172.21.15.179:0/1038443261 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f0268005570 con 0x7f02600035c0 2024-06-03T00:57:45.492 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-06-03T00:57:45.492 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","modified":"2024-06-03T00:57:39.899935Z","created":"2024-06-03T00:55:21.644119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0264039580 msgr2=0x7f026403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0264039580 0x7f026403ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f026c009180 tx=0x7f026c008040 comp rx=0 tx=0).stop 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 msgr2=0x7f0278199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 0x7f0278199e70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0278074700 tx=0x7f0268014fe0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 shutdown_connections 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027819ec80 0x7f027819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0264039580 0x7f026403ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02600035c0 0x7f0278199e70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.495 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 --2- 172.21.15.179:0/1038443261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027819a3b0 0x7f027819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:45.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 >> 172.21.15.179:0/1038443261 conn(0x7f02780fdaa0 msgr2=0x7f02781091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:45.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 shutdown_connections 2024-06-03T00:57:45.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:45.494+0000 7f027e5d7700 1 -- 172.21.15.179:0/1038443261 wait complete. 2024-06-03T00:57:45.496 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-06-03T00:57:46.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:46.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:46.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:46.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:45 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/1038443261' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:46.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:46.273 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-03T00:57:46.274 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-03T00:57:46.348 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:46.348 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:46.348 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:46.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:45 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/1038443261' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:46.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: mon.c calling monitor election 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: mon.a calling monitor election 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:47.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: monmap epoch 3 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: last_changed 2024-06-03T00:57:39.899935+0000 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: min_mon_release 19 (squid) 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: election_strategy: 1 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:47.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: fsmap 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: mgrmap e13: x(active, since 70s) 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: overall HEALTH_OK 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/1038443261' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-03T00:57:47.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:47.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: mon.b calling monitor election 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: mon.b calling monitor election 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: mon.c calling monitor election 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: mon.a calling monitor election 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: monmap epoch 3 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: last_changed 2024-06-03T00:57:39.899935+0000 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: min_mon_release 19 (squid) 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: election_strategy: 1 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-06-03T00:57:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: fsmap 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: mgrmap e13: x(active, since 72s) 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: overall HEALTH_OK 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: mon.b calling monitor election 2024-06-03T00:57:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: mon.b calling monitor election 2024-06-03T00:57:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: mon.c calling monitor election 2024-06-03T00:57:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: mon.a calling monitor election 2024-06-03T00:57:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T00:57:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: monmap epoch 3 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: last_changed 2024-06-03T00:57:39.899935+0000 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: min_mon_release 19 (squid) 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: election_strategy: 1 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: fsmap 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: mgrmap e13: x(active, since 72s) 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: overall HEALTH_OK 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: mon.b calling monitor election 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: mon.b calling monitor election 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: mon.c calling monitor election 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: mon.a calling monitor election 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: monmap epoch 3 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: last_changed 2024-06-03T00:57:39.899935+0000 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: created 2024-06-03T00:55:21.644119+0000 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: min_mon_release 19 (squid) 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: election_strategy: 1 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-06-03T00:57:48.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: fsmap 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: osdmap e4: 0 total, 0 up, 0 in 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: mgrmap e13: x(active, since 72s) 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: overall HEALTH_OK 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:48.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:48.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.760+0000 7f17fc600700 1 -- 172.21.15.94:0/2655463885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f4074830 msgr2=0x7f17f4074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:48.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.760+0000 7f17fc600700 1 --2- 172.21.15.94:0/2655463885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f4074830 0x7f17f4074c10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f17e800b0a0 tx=0x7f17e800b3b0 comp rx=0 tx=0).stop 2024-06-03T00:57:48.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.760+0000 7f17fc600700 1 -- 172.21.15.94:0/2655463885 shutdown_connections 2024-06-03T00:57:48.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.760+0000 7f17fc600700 1 --2- 172.21.15.94:0/2655463885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f4074830 0x7f17f4074c10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:48.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.760+0000 7f17fc600700 1 -- 172.21.15.94:0/2655463885 >> 172.21.15.94:0/2655463885 conn(0x7f17f40fd9a0 msgr2=0x7f17f40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:48.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.760+0000 7f17fc600700 1 -- 172.21.15.94:0/2655463885 shutdown_connections 2024-06-03T00:57:48.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.760+0000 7f17fc600700 1 -- 172.21.15.94:0/2655463885 wait complete. 2024-06-03T00:57:48.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.761+0000 7f17fc600700 1 Processor -- start 2024-06-03T00:57:48.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.761+0000 7f17fc600700 1 -- start start 2024-06-03T00:57:48.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17fc600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f4074830 0x7f17f419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:48.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17fc600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17f419e7d0 0x7f17f41a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17fc600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f41a30a0 0x7f17f41a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17fc600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17f4112ac0 con 0x7f17f41a30a0 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17fc600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17f4112940 con 0x7f17f4074830 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17fc600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f17f4112c40 con 0x7f17f419e7d0 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17f9200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17f419e7d0 0x7f17f41a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17f9c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f4074830 0x7f17f419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17fa600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f41a30a0 0x7f17f41a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17f9200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17f419e7d0 0x7f17f41a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:58192/0 (socket says 172.21.15.94:58192) 2024-06-03T00:57:48.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.762+0000 7f17f9200700 1 -- 172.21.15.94:0/2855864564 learned_addr learned my addr 172.21.15.94:0/2855864564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:57:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17f9200700 1 -- 172.21.15.94:0/2855864564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f4074830 msgr2=0x7f17f419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17f9200700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f4074830 0x7f17f419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17f9200700 1 -- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f41a30a0 msgr2=0x7f17f41a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17f9200700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f41a30a0 0x7f17f41a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17f9200700 1 -- 172.21.15.94:0/2855864564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f41aae10 con 0x7f17f419e7d0 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17f9c00700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f4074830 0x7f17f419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17fa600700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f41a30a0 0x7f17f41a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.763+0000 7f17f9200700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17f419e7d0 0x7f17f41a2b60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17e000b780 tx=0x7f17e000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.764+0000 7f17f2000700 1 -- 172.21.15.94:0/2855864564 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0016070 con 0x7f17f419e7d0 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.764+0000 7f17f2000700 1 -- 172.21.15.94:0/2855864564 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e00036a0 con 0x7f17f419e7d0 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.764+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f41aafe0 con 0x7f17f419e7d0 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.764+0000 7f17f2000700 1 -- 172.21.15.94:0/2855864564 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0011af0 con 0x7f17f419e7d0 2024-06-03T00:57:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.764+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f41ab540 con 0x7f17f419e7d0 2024-06-03T00:57:48.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.765+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f4063af0 con 0x7f17f419e7d0 2024-06-03T00:57:48.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.765+0000 7f17f2000700 1 -- 172.21.15.94:0/2855864564 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f17e0019070 con 0x7f17f419e7d0 2024-06-03T00:57:48.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.766+0000 7f17f2000700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f17dc039580 0x7f17dc03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:48.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.766+0000 7f17f2000700 1 -- 172.21.15.94:0/2855864564 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f17e004ae20 con 0x7f17f419e7d0 2024-06-03T00:57:48.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.766+0000 7f17f9c00700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f17dc039580 0x7f17dc03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:48.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.767+0000 7f17f9c00700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f17dc039580 0x7f17dc03ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17e800bfd0 tx=0x7f17e8005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:48.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.771+0000 7f17f2000700 1 -- 172.21.15.94:0/2855864564 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17e0014120 con 0x7f17f419e7d0 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:48.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.997+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f17f41ab8f0 con 0x7f17f419e7d0 2024-06-03T00:57:48.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:48.997+0000 7f17f2000700 1 -- 172.21.15.94:0/2855864564 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f17e001ad70 con 0x7f17f419e7d0 2024-06-03T00:57:48.998 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:48.998 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-06-03T00:57:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T00:57:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-06-03T00:57:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f17dc039580 msgr2=0x7f17dc03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f17dc039580 0x7f17dc03ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f17e800bfd0 tx=0x7f17e8005040 comp rx=0 tx=0).stop 2024-06-03T00:57:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17f419e7d0 msgr2=0x7f17f41a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17f419e7d0 0x7f17f41a2b60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17e000b780 tx=0x7f17e000ba90 comp rx=0 tx=0).stop 2024-06-03T00:57:49.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 shutdown_connections 2024-06-03T00:57:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17f4074830 0x7f17f419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f17dc039580 0x7f17dc03ba40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17f41a30a0 0x7f17f41a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 --2- 172.21.15.94:0/2855864564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17f419e7d0 0x7f17f41a2b60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 >> 172.21.15.94:0/2855864564 conn(0x7f17f40fd9a0 msgr2=0x7f17f40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 shutdown_connections 2024-06-03T00:57:49.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:57:49.001+0000 7f17fc600700 1 -- 172.21.15.94:0/2855864564 wait complete. 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.805 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-03T00:57:49.805 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:57:49.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T00:57:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:49 smithi094 ceph-mon[32119]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T00:57:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:49 smithi094 ceph-mon[32119]: Reconfiguring daemon mon.a on smithi094 2024-06-03T00:57:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:49 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2855864564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:49 smithi179 ceph-mon[37041]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T00:57:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:49 smithi179 ceph-mon[37041]: Reconfiguring daemon mon.a on smithi094 2024-06-03T00:57:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:49 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2855864564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:49.890 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:57:49.890 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:50.021 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:57:50.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T00:57:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:49 smithi107 ceph-mon[36622]: Reconfiguring mon.a (unknown last config time)... 2024-06-03T00:57:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:49 smithi107 ceph-mon[36622]: Reconfiguring daemon mon.a on smithi094 2024-06-03T00:57:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:49 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2855864564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:50.053 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:57:50.053 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:50.120 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:57:50.120 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-06-03T00:57:50.152 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:57:50.152 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-03T00:57:50.219 INFO:tasks.cephadm:Adding mgr.x on smithi094 2024-06-03T00:57:50.219 INFO:tasks.cephadm:Adding mgr.y on smithi107 2024-06-03T00:57:50.219 INFO:tasks.cephadm:Adding mgr.z on smithi179 2024-06-03T00:57:50.219 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi094=x;smithi107=y;smithi179=z' 2024-06-03T00:57:50.494 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T00:57:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:50 smithi094 ceph-mon[32119]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:50 smithi179 ceph-mon[37041]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:50 smithi107 ceph-mon[36622]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:52.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.029+0000 7f812e842700 1 -- 172.21.15.179:0/176918979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 msgr2=0x7f8128101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:52.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.029+0000 7f812e842700 1 --2- 172.21.15.179:0/176918979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 0x7f8128101350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8118009910 tx=0x7f8118009c20 comp rx=0 tx=0).stop 2024-06-03T00:57:52.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.029+0000 7f812e842700 1 -- 172.21.15.179:0/176918979 shutdown_connections 2024-06-03T00:57:52.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.029+0000 7f812e842700 1 --2- 172.21.15.179:0/176918979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 0x7f8128101350 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.029+0000 7f812e842700 1 -- 172.21.15.179:0/176918979 >> 172.21.15.179:0/176918979 conn(0x7f81280fc4d0 msgr2=0x7f81280fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:52.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.029+0000 7f812e842700 1 -- 172.21.15.179:0/176918979 shutdown_connections 2024-06-03T00:57:52.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.029+0000 7f812e842700 1 -- 172.21.15.179:0/176918979 wait complete. 2024-06-03T00:57:52.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 Processor -- start 2024-06-03T00:57:52.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 -- start start 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 0x7f8128197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8128197dc0 0x7f812819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128198240 0x7f812819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8128106f40 con 0x7f8128100f30 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8128106dc0 con 0x7f8128198240 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f812e842700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f81281070c0 con 0x7f8128197dc0 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f81277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8128197dc0 0x7f812819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f81277fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8128197dc0 0x7f812819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41116/0 (socket says 172.21.15.179:41116) 2024-06-03T00:57:52.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.030+0000 7f81277fe700 1 -- 172.21.15.179:0/755237161 learned_addr learned my addr 172.21.15.179:0/755237161 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81277fe700 1 -- 172.21.15.179:0/755237161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128198240 msgr2=0x7f812819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81277fe700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128198240 0x7f812819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f8127fff700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 0x7f8128197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81277fe700 1 -- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 msgr2=0x7f8128197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81277fe700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 0x7f8128197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81277fe700 1 -- 172.21.15.179:0/755237161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812819fc90 con 0x7f8128197dc0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81277fe700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8128197dc0 0x7f812819ce30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b780 tx=0x7f811c00bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81257fa700 1 -- 172.21.15.179:0/755237161 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c004230 con 0x7f8128197dc0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81257fa700 1 -- 172.21.15.179:0/755237161 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811c0043d0 con 0x7f8128197dc0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812819ff80 con 0x7f8128197dc0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f81257fa700 1 -- 172.21.15.179:0/755237161 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c011c60 con 0x7f8128197dc0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.031+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81281a0510 con 0x7f8128197dc0 2024-06-03T00:57:52.033 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.032+0000 7f81257fa700 1 -- 172.21.15.179:0/755237161 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f811c01a400 con 0x7f8128197dc0 2024-06-03T00:57:52.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.032+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8128063af0 con 0x7f8128197dc0 2024-06-03T00:57:52.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.032+0000 7f81257fa700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8110039580 0x7f811003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:57:52.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.032+0000 7f81257fa700 1 -- 172.21.15.179:0/755237161 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f811c04be30 con 0x7f8128197dc0 2024-06-03T00:57:52.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.033+0000 7f8127fff700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8110039580 0x7f811003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:57:52.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.034+0000 7f8127fff700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8110039580 0x7f811003ba40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8118009750 tx=0x7f8118005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:57:52.037 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.038+0000 7f81257fa700 1 -- 172.21.15.179:0/755237161 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f811c003c80 con 0x7f8128197dc0 2024-06-03T00:57:52.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.242+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi107=y;smithi179=z", "target": ["mon-mgr", ""]}) -- 0x7f8128062960 con 0x7f8110039580 2024-06-03T00:57:52.255 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.255+0000 7f81257fa700 1 -- 172.21.15.179:0/755237161 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8128062960 con 0x7f8110039580 2024-06-03T00:57:52.255 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8110039580 msgr2=0x7f811003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8110039580 0x7f811003ba40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8118009750 tx=0x7f8118005fb0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8128197dc0 msgr2=0x7f812819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8128197dc0 0x7f812819ce30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b780 tx=0x7f811c00bb40 comp rx=0 tx=0).stop 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 shutdown_connections 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128198240 0x7f812819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8110039580 0x7f811003ba40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8128100f30 0x7f8128197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 --2- 172.21.15.179:0/755237161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8128197dc0 0x7f812819ce30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:57:52.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 >> 172.21.15.179:0/755237161 conn(0x7f81280fc4d0 msgr2=0x7f812806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:57:52.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 shutdown_connections 2024-06-03T00:57:52.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:57:52.258+0000 7f812e842700 1 -- 172.21.15.179:0/755237161 wait complete. 2024-06-03T00:57:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:52 smithi107 ceph-mon[36622]: Reconfiguring mon.b (monmap changed)... 2024-06-03T00:57:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:52 smithi107 ceph-mon[36622]: Reconfiguring daemon mon.b on smithi107 2024-06-03T00:57:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:52 smithi094 ceph-mon[32119]: Reconfiguring mon.b (monmap changed)... 2024-06-03T00:57:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:52 smithi094 ceph-mon[32119]: Reconfiguring daemon mon.b on smithi107 2024-06-03T00:57:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:52 smithi179 ceph-mon[37041]: Reconfiguring mon.b (monmap changed)... 2024-06-03T00:57:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:52 smithi179 ceph-mon[37041]: Reconfiguring daemon mon.b on smithi107 2024-06-03T00:57:52.790 DEBUG:teuthology.orchestra.run.smithi107:mgr.y> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-03T00:57:52.792 DEBUG:teuthology.orchestra.run.smithi179:mgr.z> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-03T00:57:52.794 INFO:tasks.cephadm:Deploying OSDs... 2024-06-03T00:57:52.795 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T00:57:52.795 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T00:57:52.854 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T00:57:52.854 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-06-03T00:57:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T00:57:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:52.903 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-06-03T00:57:52.903 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:52.903 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:52.904 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-03 00:56:50.327588577 +0000 2024-06-03T00:57:52.904 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-03 00:51:32.919395207 +0000 2024-06-03T00:57:52.904 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-03 00:51:32.919395207 +0000 2024-06-03T00:57:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-03 00:51:32.919395207 +0000 2024-06-03T00:57:52.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T00:57:52.972 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-03T00:57:52.972 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-03T00:57:52.973 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000265333 s, 1.9 MB/s 2024-06-03T00:57:52.976 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T00:57:53.029 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-06-03T00:57:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T00:57:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:53.078 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-06-03T00:57:53.078 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:53.078 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:53.079 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-03 00:56:50.413587175 +0000 2024-06-03T00:57:53.079 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-03 00:51:33.474386270 +0000 2024-06-03T00:57:53.079 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-03 00:51:33.474386270 +0000 2024-06-03T00:57:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-03 00:51:33.474386270 +0000 2024-06-03T00:57:53.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T00:57:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-03T00:57:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-03T00:57:53.151 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000210371 s, 2.4 MB/s 2024-06-03T00:57:53.155 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T00:57:53.209 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-06-03T00:57:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T00:57:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:53.260 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-06-03T00:57:53.260 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:53.260 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:53.261 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-03 00:56:50.497585806 +0000 2024-06-03T00:57:53.261 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-03 00:51:34.061376817 +0000 2024-06-03T00:57:53.261 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-03 00:51:34.061376817 +0000 2024-06-03T00:57:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-03 00:51:34.061376817 +0000 2024-06-03T00:57:53.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T00:57:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-03T00:57:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-03T00:57:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.00026547 s, 1.9 MB/s 2024-06-03T00:57:53.342 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T00:57:53.396 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-06-03T00:57:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T00:57:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:53.443 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-06-03T00:57:53.444 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:53.444 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:53.444 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-03 00:56:50.577584501 +0000 2024-06-03T00:57:53.444 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-03 00:51:34.655367252 +0000 2024-06-03T00:57:53.444 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-03 00:51:34.655367252 +0000 2024-06-03T00:57:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-03 00:51:34.655367252 +0000 2024-06-03T00:57:53.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T00:57:53.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:53.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi107=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:53.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: Saving service mgr spec with placement smithi094=x;smithi107=y;smithi179=z;count:3 2024-06-03T00:57:53.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi107=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: Saving service mgr spec with placement smithi094=x;smithi107=y;smithi179=z;count:3 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:53.511 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:53.511 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:53.518 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-06-03T00:57:53.518 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-06-03T00:57:53.518 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000249662 s, 2.1 MB/s 2024-06-03T00:57:53.522 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi107=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: Saving service mgr spec with placement smithi094=x;smithi107=y;smithi179=z;count:3 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-03T00:57:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-03T00:57:53.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:53.580 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-06-03T00:57:53.580 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T00:57:53.599 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T00:57:53.599 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-06-03T00:57:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T00:57:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:53.657 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-03T00:57:53.657 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:53.657 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:53.657 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-03 00:57:39.344636594 +0000 2024-06-03T00:57:53.657 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-03 00:49:52.490347560 +0000 2024-06-03T00:57:53.657 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-03 00:49:52.490347560 +0000 2024-06-03T00:57:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-03 00:49:52.490347560 +0000 2024-06-03T00:57:53.657 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T00:57:53.723 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-03T00:57:53.723 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-03T00:57:53.723 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000226859 s, 2.3 MB/s 2024-06-03T00:57:53.724 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T00:57:53.783 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-06-03T00:57:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T00:57:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:53.840 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-03T00:57:53.840 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:53.840 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:53.841 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-03 00:57:39.344636594 +0000 2024-06-03T00:57:53.841 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-03 00:49:52.815342248 +0000 2024-06-03T00:57:53.841 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-03 00:49:52.815342248 +0000 2024-06-03T00:57:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-03 00:49:52.815342248 +0000 2024-06-03T00:57:53.841 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T00:57:53.904 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-03T00:57:53.904 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-03T00:57:53.905 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000205237 s, 2.5 MB/s 2024-06-03T00:57:53.906 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T00:57:53.963 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-06-03T00:57:54.018 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T00:57:54.018 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:54.019 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-03T00:57:54.019 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:54.019 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:54.019 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-03 00:57:39.345636578 +0000 2024-06-03T00:57:54.019 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-03 00:49:53.139336953 +0000 2024-06-03T00:57:54.019 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-03 00:49:53.139336953 +0000 2024-06-03T00:57:54.019 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-03 00:49:53.139336953 +0000 2024-06-03T00:57:54.019 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T00:57:54.082 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-03T00:57:54.083 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-03T00:57:54.083 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000226852 s, 2.3 MB/s 2024-06-03T00:57:54.084 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T00:57:54.139 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-06-03 00:57:39.345636578 +0000 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-06-03 00:49:53.480331380 +0000 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-06-03 00:49:53.480331380 +0000 2024-06-03T00:57:54.195 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-06-03 00:49:53.480331380 +0000 2024-06-03T00:57:54.195 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T00:57:54.257 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-06-03T00:57:54.257 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-06-03T00:57:54.258 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000205435 s, 2.5 MB/s 2024-06-03T00:57:54.259 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T00:57:54.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:54 smithi107 ceph-mon[36622]: Reconfiguring mon.c (monmap changed)... 2024-06-03T00:57:54.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:54 smithi107 ceph-mon[36622]: Reconfiguring daemon mon.c on smithi179 2024-06-03T00:57:54.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:54.306 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:54.315 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-06-03T00:57:54.315 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-06-03T00:57:54.329 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-03T00:57:54.330 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-03 00:57:29.344851328 +0000 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-03 00:49:52.498093853 +0000 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-03 00:49:52.498093853 +0000 2024-06-03T00:57:54.384 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-03 00:49:52.498093853 +0000 2024-06-03T00:57:54.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-03T00:57:54.448 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-03T00:57:54.448 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-03T00:57:54.448 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000214168 s, 2.4 MB/s 2024-06-03T00:57:54.449 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-03T00:57:54.505 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-03 00:57:29.345851305 +0000 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-03 00:49:52.828086148 +0000 2024-06-03T00:57:54.560 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-03 00:49:52.828086148 +0000 2024-06-03T00:57:54.561 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-03 00:49:52.828086148 +0000 2024-06-03T00:57:54.561 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-03T00:57:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:54 smithi094 ceph-mon[32119]: Reconfiguring mon.c (monmap changed)... 2024-06-03T00:57:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:54 smithi094 ceph-mon[32119]: Reconfiguring daemon mon.c on smithi179 2024-06-03T00:57:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:54 smithi179 ceph-mon[37041]: Reconfiguring mon.c (monmap changed)... 2024-06-03T00:57:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:54 smithi179 ceph-mon[37041]: Reconfiguring daemon mon.c on smithi179 2024-06-03T00:57:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:54.623 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-03T00:57:54.623 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-03T00:57:54.623 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000249832 s, 2.0 MB/s 2024-06-03T00:57:54.624 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-03T00:57:54.680 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-03 00:57:29.345851305 +0000 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-03 00:49:53.152078582 +0000 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-03 00:49:53.152078582 +0000 2024-06-03T00:57:54.734 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-03 00:49:53.152078582 +0000 2024-06-03T00:57:54.735 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-03T00:57:54.798 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-03T00:57:54.798 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-03T00:57:54.798 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000254829 s, 2.0 MB/s 2024-06-03T00:57:54.799 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-03T00:57:54.855 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-06-03T00:57:54.910 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-03T00:57:54.910 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-03T00:57:54.910 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-03T00:57:54.910 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T00:57:54.910 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-06-03T00:57:54.911 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-06-03 00:57:29.345851305 +0000 2024-06-03T00:57:54.911 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-06-03 00:49:53.487070760 +0000 2024-06-03T00:57:54.911 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-06-03 00:49:53.487070760 +0000 2024-06-03T00:57:54.911 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-06-03 00:49:53.487070760 +0000 2024-06-03T00:57:54.911 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-03T00:57:54.973 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-06-03T00:57:54.973 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-06-03T00:57:54.974 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000252464 s, 2.0 MB/s 2024-06-03T00:57:54.974 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-03T00:57:55.030 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/vg_nvme/lv_4... 2024-06-03T00:57:55.030 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T00:57:55.471 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T00:57:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:57:55.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:55.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:55 smithi107 ceph-mon[36622]: Deploying daemon mgr.z on smithi179 2024-06-03T00:57:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:55 smithi094 ceph-mon[32119]: Deploying daemon mgr.z on smithi179 2024-06-03T00:57:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:57:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:57:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:57:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T00:57:55.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:57:55.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:55.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:55 smithi179 ceph-mon[37041]: Deploying daemon mgr.z on smithi179 2024-06-03T00:57:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:56 smithi107 ceph-mon[36622]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:56.591 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:56 smithi179 ceph-mon[37041]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:56.592 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 systemd[1]: Starting Ceph mgr.z for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T00:57:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:56 smithi094 ceph-mon[32119]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:56.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 podman[38887]: 2024-06-03 00:57:56.494459371 +0000 UTC m=+0.017396251 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T00:57:56.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 podman[38887]: 2024-06-03 00:57:56.610537931 +0000 UTC m=+0.133474790 container create 729cadb4f3655de127a7150f12b79f3b87e884af190758dbe314ffade2245e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-06-03T00:57:56.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 podman[38887]: 2024-06-03 00:57:56.813019424 +0000 UTC m=+0.335956291 container init 729cadb4f3655de127a7150f12b79f3b87e884af190758dbe314ffade2245e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T00:57:56.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 podman[38887]: 2024-06-03 00:57:56.81596596 +0000 UTC m=+0.338902828 container start 729cadb4f3655de127a7150f12b79f3b87e884af190758dbe314ffade2245e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T00:57:56.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 ceph-mgr[38906]: -- 172.21.15.179:0/1690463709 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c38c42e5a0 con 0x55c38c403000 2024-06-03T00:57:57.156 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 bash[38887]: 729cadb4f3655de127a7150f12b79f3b87e884af190758dbe314ffade2245e86 2024-06-03T00:57:57.156 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 systemd[1]: Started Ceph mgr.z for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:57:57.156 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:56.972+0000 7fbceff81200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T00:57:57.156 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:57.156+0000 7fbceff81200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:57:57.560 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:57.257+0000 7fbceff81200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T00:57:57.560 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:57.560+0000 7fbceff81200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T00:57:57.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:57.644+0000 7fbceff81200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T00:57:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T00:57:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T00:57:58.248 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-03T00:57:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-03T00:57:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0363259 s, 289 MB/s 2024-06-03T00:57:58.248 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:58.249 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:57 smithi094 ceph-mon[32119]: Deploying daemon mgr.y on smithi107 2024-06-03T00:57:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:57:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T00:57:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:57:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:58.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:57 smithi107 ceph-mon[36622]: Deploying daemon mgr.y on smithi107 2024-06-03T00:57:58.289 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.289 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.289 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:57:58.289 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:57:58.290 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-03T00:57:58.290 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:57:58.290 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:57:58.290 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:57 smithi179 ceph-mon[37041]: Deploying daemon mgr.y on smithi107 2024-06-03T00:57:58.978 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:58 smithi107 systemd[1]: Starting Ceph mgr.y for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T00:57:59.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-06-03T00:57:59.229 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:57:58 smithi179 ceph-mon[37041]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:59.229 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:58.885+0000 7fbceff81200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T00:57:59.229 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:59.058+0000 7fbceff81200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T00:57:59.229 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:59.229+0000 7fbceff81200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T00:57:59.239 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:57:58 smithi107 ceph-mon[36622]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:59.239 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 podman[38000]: 2024-06-03 00:57:58.936145893 +0000 UTC m=+0.017202737 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T00:57:59.239 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 podman[38000]: 2024-06-03 00:57:59.035684329 +0000 UTC m=+0.116741157 container create 726312a8f79e95d23024c6a0ffd4018c63b70a64c475c054b800ba87f95c7a47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-03T00:57:59.239 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 podman[38000]: 2024-06-03 00:57:59.213444948 +0000 UTC m=+0.294501768 container init 726312a8f79e95d23024c6a0ffd4018c63b70a64c475c054b800ba87f95c7a47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T00:57:59.239 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 podman[38000]: 2024-06-03 00:57:59.215963924 +0000 UTC m=+0.297020749 container start 726312a8f79e95d23024c6a0ffd4018c63b70a64c475c054b800ba87f95c7a47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-03T00:57:59.240 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 ceph-mgr[38020]: -- 172.21.15.107:0/4132092025 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558db2b3c5a0 con 0x558db2b11000 2024-06-03T00:57:59.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:57:58 smithi094 ceph-mon[32119]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:57:59.466 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:57:59.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 bash[38000]: 726312a8f79e95d23024c6a0ffd4018c63b70a64c475c054b800ba87f95c7a47 2024-06-03T00:57:59.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 systemd[1]: Started Ceph mgr.y for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T00:57:59.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:57:59.366+0000 7f87e2466200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T00:57:59.837 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:59.534+0000 7fbceff81200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T00:57:59.838 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:59.626+0000 7fbceff81200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T00:57:59.838 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:57:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:57:59.838+0000 7fbceff81200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T00:57:59.981 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:57:59.552+0000 7f87e2466200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:57:59.981 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:57:59.655+0000 7f87e2466200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T00:57:59.981 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:57:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:57:59.980+0000 7f87e2466200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T00:58:00.105 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:00.035+0000 7fbceff81200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:58:00.283 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:00.068+0000 7f87e2466200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T00:58:00.625 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.625 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.626 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.626 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.626 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:00.626 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-mon[37041]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:00.628 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:00.629 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:00 smithi094 ceph-mon[32119]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:00.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:00.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:00 smithi107 ceph-mon[36622]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:01.105 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:00.671+0000 7fbceff81200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T00:58:01.407 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:01.107+0000 7fbceff81200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T00:58:01.408 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:01.203+0000 7fbceff81200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T00:58:01.408 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:01.408+0000 7fbceff81200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T00:58:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 -- 172.21.15.94:0/1543369153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc341024d0 msgr2=0x7fbc341028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 --2- 172.21.15.94:0/1543369153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc341024d0 0x7fbc341028f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc240099c0 tx=0x7fbc24009cd0 comp rx=0 tx=0).stop 2024-06-03T00:58:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 -- 172.21.15.94:0/1543369153 shutdown_connections 2024-06-03T00:58:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 --2- 172.21.15.94:0/1543369153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc341024d0 0x7fbc341028f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:01.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 --2- 172.21.15.94:0/1543369153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc34104070 0x7fbc3410a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 --2- 172.21.15.94:0/1543369153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc341036d0 0x7fbc34103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 -- 172.21.15.94:0/1543369153 >> 172.21.15.94:0/1543369153 conn(0x7fbc340fda30 msgr2=0x7fbc340ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:58:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 -- 172.21.15.94:0/1543369153 shutdown_connections 2024-06-03T00:58:01.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.617+0000 7fbc39800700 1 -- 172.21.15.94:0/1543369153 wait complete. 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.619+0000 7fbc39800700 1 Processor -- start 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.619+0000 7fbc39800700 1 -- start start 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.619+0000 7fbc39800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc341024d0 0x7fbc341987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.619+0000 7fbc39800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc341036d0 0x7fbc34198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.619+0000 7fbc39800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc34104070 0x7fbc3419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.619+0000 7fbc39800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3410e120 con 0x7fbc341024d0 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.619+0000 7fbc39800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3410dfa0 con 0x7fbc34104070 2024-06-03T00:58:01.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc39800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3410e2a0 con 0x7fbc341036d0 2024-06-03T00:58:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc341024d0 0x7fbc341987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc341036d0 0x7fbc34198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc341024d0 0x7fbc341987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45716/0 (socket says 172.21.15.94:45716) 2024-06-03T00:58:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32a00700 1 -- 172.21.15.94:0/3928184137 learned_addr learned my addr 172.21.15.94:0/3928184137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:58:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc33400700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc34104070 0x7fbc3419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32000700 1 -- 172.21.15.94:0/3928184137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc34104070 msgr2=0x7fbc3419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32000700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc34104070 0x7fbc3419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32000700 1 -- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc341024d0 msgr2=0x7fbc341987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32000700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc341024d0 0x7fbc341987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32000700 1 -- 172.21.15.94:0/3928184137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc341a03b0 con 0x7fbc341036d0 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.620+0000 7fbc32a00700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc341024d0 0x7fbc341987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.621+0000 7fbc32000700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc341036d0 0x7fbc34198ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ea20 tx=0x7fbc2000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:58:01.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.621+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000cbd0 con 0x7fbc341036d0 2024-06-03T00:58:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.621+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc20004500 con 0x7fbc341036d0 2024-06-03T00:58:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.621+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20005020 con 0x7fbc341036d0 2024-06-03T00:58:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.621+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc341a06a0 con 0x7fbc341036d0 2024-06-03T00:58:01.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.621+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341a0be0 con 0x7fbc341036d0 2024-06-03T00:58:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.622+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbc20005200 con 0x7fbc341036d0 2024-06-03T00:58:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.622+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc34069180 con 0x7fbc341036d0 2024-06-03T00:58:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.623+0000 7fbc2b400700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc18039330 0x7fbc1803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.623+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbc2004c270 con 0x7fbc341036d0 2024-06-03T00:58:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.623+0000 7fbc32a00700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc18039330 0x7fbc1803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:01.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.624+0000 7fbc32a00700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc18039330 0x7fbc1803b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc240038c0 tx=0x7fbc24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:58:01.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.628+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc20016330 con 0x7fbc341036d0 2024-06-03T00:58:01.650 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:01.314+0000 7f87e2466200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T00:58:01.650 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:01.482+0000 7f87e2466200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T00:58:01.650 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:01.649+0000 7f87e2466200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T00:58:01.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:01.860+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbc340628d0 con 0x7fbc18039330 2024-06-03T00:58:02.242 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:01.933+0000 7f87e2466200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T00:58:02.242 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:02.026+0000 7f87e2466200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T00:58:02.242 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:02.241+0000 7f87e2466200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T00:58:02.242 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.242 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:02.243 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:01 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:01.952+0000 7fbceff81200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:02.046+0000 7fbceff81200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:02.287 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.288 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:58:02.288 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:58:02.288 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.288 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:02.288 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:02.288 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:01 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:02.299 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-03T00:58:02.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-03T00:58:02.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:02.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:02.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:01 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:02.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:02.431+0000 7f87e2466200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T00:58:02.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:02.287+0000 7fbceff81200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T00:58:02.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:02.422+0000 7fbceff81200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T00:58:02.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:02.515+0000 7fbceff81200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T00:58:02.931 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:02.630+0000 7fbceff81200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T00:58:02.931 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:02.931+0000 7fbceff81200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T00:58:03.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:03 smithi094 ceph-mon[32119]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T00:58:03.165 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:03 smithi094 ceph-mon[32119]: Reconfiguring daemon mgr.x on smithi094 2024-06-03T00:58:03.166 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:03 smithi094 ceph-mon[32119]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:03.166 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:03 smithi094 ceph-mon[32119]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:03.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-mon[36622]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T00:58:03.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-mon[36622]: Reconfiguring daemon mgr.x on smithi094 2024-06-03T00:58:03.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-mon[36622]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:03.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-mon[36622]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:03.283 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:03.038+0000 7f87e2466200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T00:58:03.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:03 smithi179 ceph-mon[37041]: Reconfiguring mgr.x (unknown last config time)... 2024-06-03T00:58:03.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:03 smithi179 ceph-mon[37041]: Reconfiguring daemon mgr.x on smithi094 2024-06-03T00:58:03.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:03 smithi179 ceph-mon[37041]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:03.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:03 smithi179 ceph-mon[37041]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:03.755 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:03.456+0000 7f87e2466200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T00:58:03.756 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:03.555+0000 7f87e2466200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T00:58:03.756 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:03.755+0000 7f87e2466200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T00:58:03.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:03.533+0000 7fbceff81200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T00:58:03.855 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 00:58:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T00:58:03.753+0000 7fbceff81200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T00:58:04.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:04.369+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fbc20003680 con 0x7fbc341036d0 2024-06-03T00:58:04.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:04.277+0000 7f87e2466200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T00:58:04.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:04.370+0000 7f87e2466200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T00:58:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: Standby manager daemon z started 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:04.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.565 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: Standby manager daemon z started 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:04.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: Standby manager daemon z started 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/2499404808' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:04.957 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:04.611+0000 7f87e2466200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T00:58:04.957 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:04.747+0000 7f87e2466200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T00:58:04.957 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:04.841+0000 7f87e2466200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T00:58:04.957 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:04.958+0000 7f87e2466200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T00:58:05.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:05.254+0000 7f87e2466200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T00:58:05.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:05 smithi107 ceph-mon[36622]: mgrmap e14: x(active, since 90s), standbys: z 2024-06-03T00:58:05.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:05 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T00:58:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:05 smithi094 ceph-mon[32119]: mgrmap e14: x(active, since 90s), standbys: z 2024-06-03T00:58:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:05 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T00:58:05.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:05 smithi179 ceph-mon[37041]: mgrmap e14: x(active, since 90s), standbys: z 2024-06-03T00:58:05.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:05 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T00:58:06.283 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:05.853+0000 7f87e2466200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T00:58:06.283 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T00:58:06.068+0000 7f87e2466200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T00:58:06.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:06.385+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mon.1 v2:172.21.15.179:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fbc20014230 con 0x7fbc341036d0 2024-06-03T00:58:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: Standby manager daemon y started 2024-06-03T00:58:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T00:58:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T00:58:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T00:58:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T00:58:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:06.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3336032447' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63946c12-7fdf-401c-9cbc-18db7ef2f714"}]: dispatch 2024-06-03T00:58:06.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3336032447' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63946c12-7fdf-401c-9cbc-18db7ef2f714"}]': finished 2024-06-03T00:58:06.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T00:58:06.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:06 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: Standby manager daemon y started 2024-06-03T00:58:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T00:58:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T00:58:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T00:58:06.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T00:58:06.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:06.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3336032447' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63946c12-7fdf-401c-9cbc-18db7ef2f714"}]: dispatch 2024-06-03T00:58:06.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3336032447' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63946c12-7fdf-401c-9cbc-18db7ef2f714"}]': finished 2024-06-03T00:58:06.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T00:58:06.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:06 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: Standby manager daemon y started 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/1717916355' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3336032447' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63946c12-7fdf-401c-9cbc-18db7ef2f714"}]: dispatch 2024-06-03T00:58:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3336032447' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63946c12-7fdf-401c-9cbc-18db7ef2f714"}]': finished 2024-06-03T00:58:06.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: osdmap e5: 1 total, 0 up, 1 in 2024-06-03T00:58:06.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:06 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:07 smithi094 ceph-mon[32119]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-06-03T00:58:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T00:58:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:07 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2587385698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:58:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:07 smithi107 ceph-mon[36622]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-06-03T00:58:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T00:58:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:07 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2587385698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:58:07.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:07 smithi179 ceph-mon[37041]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-06-03T00:58:07.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T00:58:07.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:07 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2587385698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:58:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:08 smithi107 ceph-mon[36622]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:08.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:08 smithi094 ceph-mon[32119]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:08 smithi179 ceph-mon[37041]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:11.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:11 smithi094 ceph-mon[32119]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:11.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:11 smithi107 ceph-mon[36622]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:11 smithi179 ceph-mon[37041]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:13.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:13 smithi107 ceph-mon[36622]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:13 smithi094 ceph-mon[32119]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:13 smithi179 ceph-mon[37041]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:15.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:15 smithi094 ceph-mon[32119]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:15 smithi107 ceph-mon[36622]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:15 smithi179 ceph-mon[37041]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:16.473 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T00:58:16.474 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:16.506 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T00:58:16.506 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-03T00:58:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:17.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:17 smithi094 ceph-mon[32119]: Deploying daemon osd.0 on smithi094 2024-06-03T00:58:17.485 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:17 smithi094 ceph-mon[32119]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:17 smithi107 ceph-mon[36622]: Deploying daemon osd.0 on smithi094 2024-06-03T00:58:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:17 smithi107 ceph-mon[36622]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:17 smithi179 ceph-mon[37041]: Deploying daemon osd.0 on smithi094 2024-06-03T00:58:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:17 smithi179 ceph-mon[37041]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:19.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:19 smithi094 ceph-mon[32119]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:19 smithi107 ceph-mon[36622]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:19 smithi179 ceph-mon[37041]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:21.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:21 smithi094 ceph-mon[32119]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:21.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:21 smithi107 ceph-mon[36622]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:21 smithi179 ceph-mon[37041]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:23.386 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:23 smithi094 ceph-mon[32119]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:23 smithi107 ceph-mon[36622]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:23 smithi179 ceph-mon[37041]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:24.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:24.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:24.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:24 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:24.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:24 smithi094 ceph-mon[32119]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:24 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:24 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:24 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:24 smithi179 ceph-mon[37041]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:25.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:24 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:25.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:24 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:25.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:24 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:25.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:24 smithi107 ceph-mon[36622]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:27.512 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:27 smithi094 ceph-mon[32119]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:27.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:27.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:27.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:27 smithi107 ceph-mon[36622]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:27 smithi179 ceph-mon[37041]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:28.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.406+0000 7fbc2b400700 1 -- 172.21.15.94:0/3928184137 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbc340628d0 con 0x7fbc18039330 2024-06-03T00:58:28.408 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-06-03T00:58:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.409+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc18039330 msgr2=0x7fbc1803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.409+0000 7fbc39800700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc18039330 0x7fbc1803b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc240038c0 tx=0x7fbc24005fb0 comp rx=0 tx=0).stop 2024-06-03T00:58:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.409+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc341036d0 msgr2=0x7fbc34198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.409+0000 7fbc39800700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc341036d0 0x7fbc34198ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ea20 tx=0x7fbc2000ed30 comp rx=0 tx=0).stop 2024-06-03T00:58:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 shutdown_connections 2024-06-03T00:58:28.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc34104070 0x7fbc3419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc18039330 0x7fbc1803b7f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc341024d0 0x7fbc341987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 --2- 172.21.15.94:0/3928184137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc341036d0 0x7fbc34198ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 >> 172.21.15.94:0/3928184137 conn(0x7fbc340fda30 msgr2=0x7fbc34106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:58:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 shutdown_connections 2024-06-03T00:58:28.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:58:28.410+0000 7fbc39800700 1 -- 172.21.15.94:0/3928184137 wait complete. 2024-06-03T00:58:28.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:28 smithi094 ceph-mon[32119]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T00:58:28.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:28 smithi107 ceph-mon[36622]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T00:58:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:28 smithi179 ceph-mon[37041]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-03T00:58:29.211 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.0.service 2024-06-03T00:58:29.216 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_4... 2024-06-03T00:58:29.216 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T00:58:29.452 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:29.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T00:58:29.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T00:58:29.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:58:29.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:29.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:29.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:29.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:29.455 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T00:58:29.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:29.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T00:58:29.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T00:58:29.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:58:29.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:29.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:29.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:29.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:29 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-03T00:58:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: osdmap e6: 1 total, 0 up, 1 in 2024-06-03T00:58:29.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:58:29.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:29.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:29.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:29.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:29 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:30.100 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0[47187]: 2024-06-03T00:58:29.929+0000 7f0c84a00700 -1 osd.0 0 waiting for initial osdmap 2024-06-03T00:58:30.100 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:58:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0[47187]: 2024-06-03T00:58:29.947+0000 7f0c7a800700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T00:58:30.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:30 smithi094 ceph-mon[32119]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T00:58:30.478 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:30 smithi094 ceph-mon[32119]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T00:58:30.479 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:30 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:30.479 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:30 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:30.479 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:30 smithi094 ceph-mon[32119]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' 2024-06-03T00:58:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:30 smithi107 ceph-mon[36622]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T00:58:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:30 smithi107 ceph-mon[36622]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T00:58:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:30 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:30 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:30 smithi107 ceph-mon[36622]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' 2024-06-03T00:58:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:30 smithi179 ceph-mon[37041]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T00:58:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:30 smithi179 ceph-mon[37041]: osdmap e7: 1 total, 0 up, 1 in 2024-06-03T00:58:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:30 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:30 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:30.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:30 smithi179 ceph-mon[37041]: from='osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984]' entity='osd.0' 2024-06-03T00:58:31.110 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T00:58:31.110 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T00:58:31.110 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-03T00:58:31.110 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-03T00:58:31.111 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112206 s, 935 MB/s 2024-06-03T00:58:31.111 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-03T00:58:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:31 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T00:58:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:31 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T00:58:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:31 smithi094 ceph-mon[32119]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:31 smithi094 ceph-mon[32119]: osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] boot 2024-06-03T00:58:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:31 smithi094 ceph-mon[32119]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T00:58:31.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:31.404 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:31 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T00:58:31.404 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:31 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T00:58:31.404 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:31 smithi107 ceph-mon[36622]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:31.404 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:31.404 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:31 smithi107 ceph-mon[36622]: osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] boot 2024-06-03T00:58:31.405 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:31 smithi107 ceph-mon[36622]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T00:58:31.405 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:31 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T00:58:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:31 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T00:58:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:31 smithi179 ceph-mon[37041]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-03T00:58:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:31.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:31 smithi179 ceph-mon[37041]: osd.0 [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] boot 2024-06-03T00:58:31.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:31 smithi179 ceph-mon[37041]: osdmap e8: 1 total, 1 up, 1 in 2024-06-03T00:58:31.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T00:58:31.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-06-03T00:58:32.031 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T00:58:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 -- 172.21.15.107:0/906309772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a501036f0 msgr2=0x7f9a50103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 --2- 172.21.15.107:0/906309772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a501036f0 0x7f9a50103b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9a440099c0 tx=0x7f9a44009cd0 comp rx=0 tx=0).stop 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 -- 172.21.15.107:0/906309772 shutdown_connections 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 --2- 172.21.15.107:0/906309772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a50104090 0x7f9a5010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 --2- 172.21.15.107:0/906309772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501024f0 0x7f9a50102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 --2- 172.21.15.107:0/906309772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a501036f0 0x7f9a50103b50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 -- 172.21.15.107:0/906309772 >> 172.21.15.107:0/906309772 conn(0x7f9a500fda90 msgr2=0x7f9a500ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 -- 172.21.15.107:0/906309772 shutdown_connections 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.465+0000 7f9a56283700 1 -- 172.21.15.107:0/906309772 wait complete. 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 Processor -- start 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 -- start start 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a501024f0 0x7f9a50076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501036f0 0x7f9a50074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:33.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a50104090 0x7f9a50074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010e130 con 0x7f9a501024f0 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010dfb0 con 0x7f9a50104090 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a56283700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010e2b0 con 0x7f9a501036f0 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a54820700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a50104090 0x7f9a50074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a54820700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a50104090 0x7f9a50074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53356/0 (socket says 172.21.15.107:53356) 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a54820700 1 -- 172.21.15.107:0/583019340 learned_addr learned my addr 172.21.15.107:0/583019340 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a4f7fe700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501036f0 0x7f9a50074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.466+0000 7f9a4ffff700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a501024f0 0x7f9a50076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a54820700 1 -- 172.21.15.107:0/583019340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501036f0 msgr2=0x7f9a50074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a54820700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501036f0 0x7f9a50074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:33.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a54820700 1 -- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a501024f0 msgr2=0x7f9a50076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:33.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a54820700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a501024f0 0x7f9a50076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:33.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a54820700 1 -- 172.21.15.107:0/583019340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a50075670 con 0x7f9a50104090 2024-06-03T00:58:33.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a4ffff700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a501024f0 0x7f9a50076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T00:58:33.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a4f7fe700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501036f0 0x7f9a50074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T00:58:33.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a54820700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a50104090 0x7f9a50074dc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b970 tx=0x7f9a4800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:58:33.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a4d7fa700 1 -- 172.21.15.107:0/583019340 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48019070 con 0x7f9a50104090 2024-06-03T00:58:33.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a4d7fa700 1 -- 172.21.15.107:0/583019340 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a4800ce10 con 0x7f9a50104090 2024-06-03T00:58:33.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a501a28e0 con 0x7f9a50104090 2024-06-03T00:58:33.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a4d7fa700 1 -- 172.21.15.107:0/583019340 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48007370 con 0x7f9a50104090 2024-06-03T00:58:33.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.467+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a501a2e70 con 0x7f9a50104090 2024-06-03T00:58:33.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.468+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a50069180 con 0x7f9a50104090 2024-06-03T00:58:33.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.469+0000 7f9a4d7fa700 1 -- 172.21.15.107:0/583019340 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f9a4801c070 con 0x7f9a50104090 2024-06-03T00:58:33.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.469+0000 7f9a4d7fa700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9a380a2c10 0x7f9a380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:33.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.470+0000 7f9a4d7fa700 1 -- 172.21.15.107:0/583019340 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f9a480cc8e0 con 0x7f9a50104090 2024-06-03T00:58:33.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.470+0000 7f9a4ffff700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9a380a2c10 0x7f9a380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:33.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.471+0000 7f9a4ffff700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9a380a2c10 0x7f9a380a50d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9a40005fd0 tx=0x7f9a40005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:58:33.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.474+0000 7f9a4d7fa700 1 -- 172.21.15.107:0/583019340 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a4809b2b0 con 0x7f9a50104090 2024-06-03T00:58:33.475 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:33 smithi107 ceph-mon[36622]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:33.475 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:33 smithi107 ceph-mon[36622]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T00:58:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:33 smithi094 ceph-mon[32119]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:33.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:33 smithi094 ceph-mon[32119]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T00:58:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:33 smithi179 ceph-mon[37041]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:33 smithi179 ceph-mon[37041]: osdmap e9: 1 total, 1 up, 1 in 2024-06-03T00:58:33.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:33.677+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9a500628d0 con 0x7f9a380a2c10 2024-06-03T00:58:34.506 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:34.507 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:58:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:35.460 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:35 smithi107 ceph-mon[36622]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:35.460 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:35 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi094 2024-06-03T00:58:35.460 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:35 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi094 to 17089M 2024-06-03T00:58:35.460 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:35 smithi107 ceph-mon[36622]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:35 smithi094 ceph-mon[32119]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:35 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi094 2024-06-03T00:58:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:35 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi094 to 17089M 2024-06-03T00:58:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:35 smithi094 ceph-mon[32119]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:35 smithi179 ceph-mon[37041]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:35 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi094 2024-06-03T00:58:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:35 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi094 to 17089M 2024-06-03T00:58:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:35 smithi179 ceph-mon[37041]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:37 smithi107 ceph-mon[36622]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:37 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/1760469591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]: dispatch 2024-06-03T00:58:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:37 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]: dispatch 2024-06-03T00:58:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:37 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]': finished 2024-06-03T00:58:37.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:37 smithi107 ceph-mon[36622]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T00:58:37.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:37.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:37 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/399340953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:58:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:37 smithi094 ceph-mon[32119]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:37 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/1760469591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]: dispatch 2024-06-03T00:58:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:37 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]: dispatch 2024-06-03T00:58:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:37 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]': finished 2024-06-03T00:58:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:37 smithi094 ceph-mon[32119]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T00:58:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:37.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:37 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/399340953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:58:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:37 smithi179 ceph-mon[37041]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:37 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/1760469591' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]: dispatch 2024-06-03T00:58:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:37 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]: dispatch 2024-06-03T00:58:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:37 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e"}]': finished 2024-06-03T00:58:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:37 smithi179 ceph-mon[37041]: osdmap e10: 2 total, 1 up, 2 in 2024-06-03T00:58:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:37 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/399340953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:58:39.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:39 smithi094 ceph-mon[32119]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:39 smithi107 ceph-mon[36622]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:39.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:39 smithi179 ceph-mon[37041]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:41.395 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:41 smithi107 ceph-mon[36622]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:41.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:41 smithi094 ceph-mon[32119]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:41.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:41 smithi179 ceph-mon[37041]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:43.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:43 smithi107 ceph-mon[36622]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:43 smithi094 ceph-mon[32119]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:43 smithi179 ceph-mon[37041]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T00:58:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T00:58:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-03T00:58:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:45.520 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:45 smithi107 ceph-mon[36622]: Deploying daemon osd.1 on smithi107 2024-06-03T00:58:45.521 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:45 smithi107 ceph-mon[36622]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:45 smithi094 ceph-mon[32119]: Deploying daemon osd.1 on smithi107 2024-06-03T00:58:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:45 smithi094 ceph-mon[32119]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:45 smithi179 ceph-mon[37041]: Deploying daemon osd.1 on smithi107 2024-06-03T00:58:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:45 smithi179 ceph-mon[37041]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:46.445 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:46 smithi107 ceph-mon[36622]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:46 smithi094 ceph-mon[32119]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:46.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:46 smithi179 ceph-mon[37041]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:49.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:49 smithi107 ceph-mon[36622]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:49.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:49.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:49.251 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:49 smithi094 ceph-mon[32119]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:49 smithi179 ceph-mon[37041]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:51.472 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:51 smithi107 ceph-mon[36622]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:51.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.562+0000 7f9a4d7fa700 1 -- 172.21.15.107:0/583019340 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9a500628d0 con 0x7f9a380a2c10 2024-06-03T00:58:51.563 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-06-03T00:58:51.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9a380a2c10 msgr2=0x7f9a380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:51.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9a380a2c10 0x7f9a380a50d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9a40005fd0 tx=0x7f9a40005dc0 comp rx=0 tx=0).stop 2024-06-03T00:58:51.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a50104090 msgr2=0x7f9a50074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a50104090 0x7f9a50074dc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800b970 tx=0x7f9a4800bc80 comp rx=0 tx=0).stop 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 shutdown_connections 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a50104090 0x7f9a50074dc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9a380a2c10 0x7f9a380a50d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9a501024f0 0x7f9a50076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 --2- 172.21.15.107:0/583019340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a501036f0 0x7f9a50074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.564+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 >> 172.21.15.107:0/583019340 conn(0x7f9a500fda90 msgr2=0x7f9a50106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.565+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 shutdown_connections 2024-06-03T00:58:51.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:58:51.565+0000 7f9a56283700 1 -- 172.21.15.107:0/583019340 wait complete. 2024-06-03T00:58:51.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:51 smithi094 ceph-mon[32119]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:51 smithi179 ceph-mon[37041]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:52.563 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.1.service 2024-06-03T00:58:52.566 INFO:tasks.cephadm:Deploying osd.2 on smithi179 with /dev/vg_nvme/lv_4... 2024-06-03T00:58:52.566 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-03T00:58:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:52 smithi107 ceph-mon[36622]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:52 smithi107 ceph-mon[36622]: from='osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T00:58:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:52 smithi107 ceph-mon[36622]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T00:58:52.806 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T00:58:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:52 smithi094 ceph-mon[32119]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:52 smithi094 ceph-mon[32119]: from='osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T00:58:52.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:52 smithi094 ceph-mon[32119]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T00:58:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:52 smithi179 ceph-mon[37041]: pgmap v69: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:52 smithi179 ceph-mon[37041]: from='osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T00:58:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:52 smithi179 ceph-mon[37041]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-03T00:58:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:54 smithi107 ceph-mon[36622]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T00:58:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:54 smithi107 ceph-mon[36622]: from='osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T00:58:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:54 smithi107 ceph-mon[36622]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T00:58:54.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:54.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:54 smithi107 ceph-mon[36622]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T00:58:54.534 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:58:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1[41190]: 2024-06-03T00:58:54.216+0000 7f90185b6700 -1 osd.1 0 waiting for initial osdmap 2024-06-03T00:58:54.534 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:58:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1[41190]: 2024-06-03T00:58:54.219+0000 7f90143a9700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T00:58:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:54 smithi094 ceph-mon[32119]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T00:58:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:54 smithi094 ceph-mon[32119]: from='osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T00:58:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:54 smithi094 ceph-mon[32119]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T00:58:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:54 smithi094 ceph-mon[32119]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T00:58:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:54 smithi179 ceph-mon[37041]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-03T00:58:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:54 smithi179 ceph-mon[37041]: from='osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T00:58:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:54 smithi179 ceph-mon[37041]: osdmap e11: 2 total, 1 up, 2 in 2024-06-03T00:58:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:54 smithi179 ceph-mon[37041]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T00:58:54.751 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-03T00:58:54.751 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-03T00:58:54.751 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-03T00:58:54.751 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-03T00:58:54.752 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114103 s, 919 MB/s 2024-06-03T00:58:54.752 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 2024-06-03T00:58:54.752 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-03T00:58:55.333 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:55 smithi179 ceph-mon[37041]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:55.333 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:55 smithi179 ceph-mon[37041]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T00:58:55.333 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:55 smithi179 ceph-mon[37041]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T00:58:55.333 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:55.333 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:55.360 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-06-03T00:58:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:55 smithi107 ceph-mon[36622]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:55 smithi107 ceph-mon[36622]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T00:58:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:55 smithi107 ceph-mon[36622]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T00:58:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:55.589 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T00:58:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:55 smithi094 ceph-mon[32119]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:55 smithi094 ceph-mon[32119]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T00:58:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:55 smithi094 ceph-mon[32119]: osdmap e12: 2 total, 1 up, 2 in 2024-06-03T00:58:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:55.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T00:58:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T00:58:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] boot 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] boot 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:56.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:56.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:56.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:56.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: osd.1 [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] boot 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: osdmap e13: 2 total, 2 up, 2 in 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:58:56.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:58:57.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 -- 172.21.15.179:0/3866919304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 msgr2=0x7f290c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:57.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 --2- 172.21.15.179:0/3866919304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c101840 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009990 tx=0x7f28fc009ca0 comp rx=0 tx=0).stop 2024-06-03T00:58:57.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 -- 172.21.15.179:0/3866919304 shutdown_connections 2024-06-03T00:58:57.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 --2- 172.21.15.179:0/3866919304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f290c102fc0 0x7f290c109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:57.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 --2- 172.21.15.179:0/3866919304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c101840 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:57.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 --2- 172.21.15.179:0/3866919304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290c102620 0x7f290c102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:57.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 -- 172.21.15.179:0/3866919304 >> 172.21.15.179:0/3866919304 conn(0x7f290c0fc9e0 msgr2=0x7f290c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 -- 172.21.15.179:0/3866919304 shutdown_connections 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.128+0000 7f2911f5a700 1 -- 172.21.15.179:0/3866919304 wait complete. 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 Processor -- start 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 -- start start 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f290c102620 0x7f290c19d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290c102fc0 0x7f290c19fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f290c10d060 con 0x7f290c102fc0 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f290c10cee0 con 0x7f290c102620 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f2911f5a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f290c10d1e0 con 0x7f290c101420 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:57.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c06b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33680/0 (socket says 172.21.15.179:33680) 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 -- 172.21.15.179:0/2720934386 learned_addr learned my addr 172.21.15.179:0/2720934386 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290affd700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f290c102620 0x7f290c19d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 -- 172.21.15.179:0/2720934386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f290c102620 msgr2=0x7f290c19d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f290c102620 0x7f290c19d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 -- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290c102fc0 msgr2=0x7f290c19fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290c102fc0 0x7f290c19fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.129+0000 7f290b7fe700 1 -- 172.21.15.179:0/2720934386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290c1a03f0 con 0x7f290c101420 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.130+0000 7f290b7fe700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c06b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0039c0 tx=0x7f28fc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.130+0000 7f2908ff9700 1 -- 172.21.15.179:0/2720934386 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc003c70 con 0x7f290c101420 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.130+0000 7f2908ff9700 1 -- 172.21.15.179:0/2720934386 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28fc004830 con 0x7f290c101420 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.130+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290c1a0680 con 0x7f290c101420 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.130+0000 7f2908ff9700 1 -- 172.21.15.179:0/2720934386 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc005440 con 0x7f290c101420 2024-06-03T00:58:57.133 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.130+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f290c1a0ad0 con 0x7f290c101420 2024-06-03T00:58:57.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.131+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290c02dca0 con 0x7f290c101420 2024-06-03T00:58:57.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.132+0000 7f2908ff9700 1 -- 172.21.15.179:0/2720934386 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f28fc003e10 con 0x7f290c101420 2024-06-03T00:58:57.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.132+0000 7f2908ff9700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f28f40a2c10 0x7f28f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:58:57.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.132+0000 7f2908ff9700 1 -- 172.21.15.179:0/2720934386 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f28fc0ce820 con 0x7f290c101420 2024-06-03T00:58:57.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.133+0000 7f290affd700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f28f40a2c10 0x7f28f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:58:57.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.134+0000 7f290affd700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f28f40a2c10 0x7f28f40a50d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f290c06bc40 tx=0x7f2900009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:58:57.137 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.137+0000 7f2908ff9700 1 -- 172.21.15.179:0/2720934386 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f28fc013070 con 0x7f290c101420 2024-06-03T00:58:57.343 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:58:57.342+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f290c0628d0 con 0x7f28f40a2c10 2024-06-03T00:58:57.343 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:57 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi107 2024-06-03T00:58:57.343 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:57 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi107 to 17088M 2024-06-03T00:58:57.343 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:57 smithi179 ceph-mon[37041]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:57.343 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:57 smithi179 ceph-mon[37041]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T00:58:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:57 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi107 2024-06-03T00:58:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:57 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi107 to 17088M 2024-06-03T00:58:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:57 smithi107 ceph-mon[36622]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:57 smithi107 ceph-mon[36622]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T00:58:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:57 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi107 2024-06-03T00:58:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:57 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi107 to 17088M 2024-06-03T00:58:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:57 smithi094 ceph-mon[32119]: pgmap v74: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-03T00:58:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:57 smithi094 ceph-mon[32119]: osdmap e14: 2 total, 2 up, 2 in 2024-06-03T00:58:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:58:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:58:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:58:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:59 smithi107 ceph-mon[36622]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:58:59 smithi107 ceph-mon[36622]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:58:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:59 smithi094 ceph-mon[32119]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:58:59 smithi094 ceph-mon[32119]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:58:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:59 smithi179 ceph-mon[37041]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:58:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:58:59 smithi179 ceph-mon[37041]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:00 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/2475563862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]: dispatch 2024-06-03T00:59:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:00 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]: dispatch 2024-06-03T00:59:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:00 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]': finished 2024-06-03T00:59:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:00 smithi107 ceph-mon[36622]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T00:59:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:00 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/2475563862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]: dispatch 2024-06-03T00:59:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:00 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]: dispatch 2024-06-03T00:59:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:00 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]': finished 2024-06-03T00:59:00.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:00 smithi094 ceph-mon[32119]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T00:59:00.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:00 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/2475563862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]: dispatch 2024-06-03T00:59:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:00 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]: dispatch 2024-06-03T00:59:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:00 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1f9c436-0127-4d60-9cfa-4a33e22409f4"}]': finished 2024-06-03T00:59:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:00 smithi179 ceph-mon[37041]: osdmap e15: 3 total, 2 up, 3 in 2024-06-03T00:59:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:01.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:01 smithi107 ceph-mon[36622]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:01.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:01 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/1789288243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:59:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:01 smithi094 ceph-mon[32119]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:01 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/1789288243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:59:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:01 smithi179 ceph-mon[37041]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:01 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/1789288243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:59:03.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:03 smithi107 ceph-mon[36622]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:03.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:03 smithi094 ceph-mon[32119]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:03 smithi179 ceph-mon[37041]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:04 smithi094 ceph-mon[32119]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:04 smithi179 ceph-mon[37041]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:04 smithi107 ceph-mon[36622]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:07 smithi107 ceph-mon[36622]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T00:59:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:07 smithi094 ceph-mon[32119]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T00:59:07.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:07 smithi179 ceph-mon[37041]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-03T00:59:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:08.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:08 smithi179 ceph-mon[37041]: Deploying daemon osd.2 on smithi179 2024-06-03T00:59:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:08 smithi107 ceph-mon[36622]: Deploying daemon osd.2 on smithi179 2024-06-03T00:59:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:08 smithi094 ceph-mon[32119]: Deploying daemon osd.2 on smithi179 2024-06-03T00:59:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:09 smithi107 ceph-mon[36622]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:09 smithi094 ceph-mon[32119]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:09 smithi179 ceph-mon[37041]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:11.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:11 smithi179 ceph-mon[37041]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:11.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:11 smithi107 ceph-mon[36622]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:11 smithi094 ceph-mon[32119]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:13.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:13 smithi179 ceph-mon[37041]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:13.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:13.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:13.522 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:13.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:13 smithi107 ceph-mon[36622]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:13.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:13.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:13.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:13 smithi094 ceph-mon[32119]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:15 smithi107 ceph-mon[36622]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:15 smithi179 ceph-mon[37041]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:15.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:15.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:15.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:15 smithi094 ceph-mon[32119]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:16.278 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.278+0000 7f2908ff9700 1 -- 172.21.15.179:0/2720934386 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f290c0628d0 con 0x7f28f40a2c10 2024-06-03T00:59:16.279 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 2 on host 'smithi179' 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f28f40a2c10 msgr2=0x7f28f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f28f40a2c10 0x7f28f40a50d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f290c06bc40 tx=0x7f2900009190 comp rx=0 tx=0).stop 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 msgr2=0x7f290c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c06b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0039c0 tx=0x7f28fc0047b0 comp rx=0 tx=0).stop 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 shutdown_connections 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f28f40a2c10 0x7f28f40a50d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290c102fc0 0x7f290c19fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f290c101420 0x7f290c06b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 --2- 172.21.15.179:0/2720934386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f290c102620 0x7f290c19d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:16.281 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 >> 172.21.15.179:0/2720934386 conn(0x7f290c0fc9e0 msgr2=0x7f290c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:59:16.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 shutdown_connections 2024-06-03T00:59:16.282 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T00:59:16.281+0000 7f2911f5a700 1 -- 172.21.15.179:0/2720934386 wait complete. 2024-06-03T00:59:16.900 DEBUG:teuthology.orchestra.run.smithi179:osd.2> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.2.service 2024-06-03T00:59:16.902 INFO:tasks.cephadm:Deploying osd.3 on smithi094 with /dev/vg_nvme/lv_3... 2024-06-03T00:59:16.903 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T00:59:17.335 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:59:17.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:17 smithi094 ceph-mon[32119]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:17.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:17 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:17.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:17 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:17.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:17 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:17.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:17 smithi094 ceph-mon[32119]: from='osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T00:59:17.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:17 smithi094 ceph-mon[32119]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T00:59:17.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:17 smithi179 ceph-mon[37041]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:17.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:17 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:17.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:17 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:17.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:17 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:17.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:17 smithi179 ceph-mon[37041]: from='osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T00:59:17.443 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:17 smithi179 ceph-mon[37041]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T00:59:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:17 smithi107 ceph-mon[36622]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:17 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:17 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:17 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:17 smithi107 ceph-mon[36622]: from='osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T00:59:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:17 smithi107 ceph-mon[36622]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-03T00:59:18.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:18 smithi179 ceph-mon[37041]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T00:59:18.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:18 smithi179 ceph-mon[37041]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T00:59:18.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:18 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:18.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:18 smithi179 ceph-mon[37041]: from='osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T00:59:18.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:18 smithi179 ceph-mon[37041]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T00:59:18.390 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2[42154]: 2024-06-03T00:59:18.254+0000 7efc8c7b9700 -1 osd.2 0 waiting for initial osdmap 2024-06-03T00:59:18.390 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2[42154]: 2024-06-03T00:59:18.261+0000 7efc885ac700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T00:59:18.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:18 smithi094 ceph-mon[32119]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T00:59:18.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:18 smithi094 ceph-mon[32119]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T00:59:18.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:18 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:18.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:18 smithi094 ceph-mon[32119]: from='osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T00:59:18.507 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:18 smithi094 ceph-mon[32119]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T00:59:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:18 smithi107 ceph-mon[36622]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-03T00:59:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:18 smithi107 ceph-mon[36622]: osdmap e16: 3 total, 2 up, 3 in 2024-06-03T00:59:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:18 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:18.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:18 smithi107 ceph-mon[36622]: from='osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T00:59:18.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:18 smithi107 ceph-mon[36622]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T00:59:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:19 smithi107 ceph-mon[36622]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:19 smithi107 ceph-mon[36622]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T00:59:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:19 smithi107 ceph-mon[36622]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T00:59:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:19 smithi094 ceph-mon[32119]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:19 smithi094 ceph-mon[32119]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T00:59:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:19 smithi094 ceph-mon[32119]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T00:59:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:19 smithi179 ceph-mon[37041]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:19 smithi179 ceph-mon[37041]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T00:59:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:19 smithi179 ceph-mon[37041]: osdmap e17: 3 total, 2 up, 3 in 2024-06-03T00:59:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T00:59:20.149 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T00:59:20.149 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-03T00:59:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-03T00:59:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0338682 s, 310 MB/s 2024-06-03T00:59:20.150 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 2024-06-03T00:59:20.150 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-03T00:59:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T00:59:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T00:59:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] boot 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:20.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.010 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-06-03T00:59:21.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T00:59:21.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T00:59:21.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:21.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] boot 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:21.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T00:59:21.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T00:59:21.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: osd.2 [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] boot 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: osdmap e18: 3 total, 3 up, 3 in 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:21.439 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T00:59:21.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:21 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi179 2024-06-03T00:59:21.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:21 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi179 to 17088M 2024-06-03T00:59:21.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:21 smithi094 ceph-mon[32119]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:21.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T00:59:21.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T00:59:21.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:21 smithi094 ceph-mon[32119]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T00:59:21.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T00:59:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:21 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi179 2024-06-03T00:59:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:21 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi179 to 17088M 2024-06-03T00:59:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:21 smithi179 ceph-mon[37041]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T00:59:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T00:59:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:21 smithi179 ceph-mon[37041]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T00:59:22.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T00:59:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:21 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi179 2024-06-03T00:59:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:21 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi179 to 17088M 2024-06-03T00:59:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:21 smithi107 ceph-mon[36622]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-03T00:59:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-03T00:59:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-03T00:59:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:21 smithi107 ceph-mon[36622]: osdmap e19: 3 total, 3 up, 3 in 2024-06-03T00:59:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T00:59:22.815 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:22 smithi094 sudo[51524]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T00:59:22.815 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:22 smithi094 sudo[51524]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:22.816 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:22 smithi094 sudo[51524]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:22.816 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:22 smithi094 sudo[51524]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:23.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:22 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T00:59:23.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:22 smithi094 ceph-mon[32119]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T00:59:23.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:22 smithi094 ceph-mon[32119]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:23.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:22 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T00:59:23.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:22 smithi179 ceph-mon[37041]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T00:59:23.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:22 smithi179 ceph-mon[37041]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:23.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:22 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-03T00:59:23.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:22 smithi107 ceph-mon[36622]: osdmap e20: 3 total, 3 up, 3 in 2024-06-03T00:59:23.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:22 smithi107 ceph-mon[36622]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:23.365 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:23 smithi094 sudo[51547]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T00:59:23.365 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:23 smithi094 sudo[51547]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:23.365 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:23 smithi094 sudo[51547]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:23.365 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 00:59:23 smithi094 sudo[51547]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:23.643 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45178]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T00:59:23.643 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:23.643 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45178]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:23.643 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45178]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 -- 172.21.15.94:0/3244605803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 msgr2=0x7fa968074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3244605803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 0x7fa968074920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00b0a0 tx=0x7fa95c00b3b0 comp rx=0 tx=0).stop 2024-06-03T00:59:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 -- 172.21.15.94:0/3244605803 shutdown_connections 2024-06-03T00:59:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3244605803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa96810eae0 0x7fa968110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3244605803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 0x7fa968074920 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3244605803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa968074ef0 0x7fa968075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 -- 172.21.15.94:0/3244605803 >> 172.21.15.94:0/3244605803 conn(0x7fa9680fda60 msgr2=0x7fa9680ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:59:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 -- 172.21.15.94:0/3244605803 shutdown_connections 2024-06-03T00:59:23.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.691+0000 7fa96fe00700 1 -- 172.21.15.94:0/3244605803 wait complete. 2024-06-03T00:59:23.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.692+0000 7fa96fe00700 1 Processor -- start 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.692+0000 7fa96fe00700 1 -- start start 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa968074ef0 0x7fa96819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96fe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 0x7fa96819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa96810eae0 0x7fa9681a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96fe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa968114710 con 0x7fa968074ef0 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa968114590 con 0x7fa96810eae0 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96fe00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa968114890 con 0x7fa9680764c0 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa96810eae0 0x7fa9681a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96d400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa968074ef0 0x7fa96819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa96810eae0 0x7fa9681a6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:34538/0 (socket says 172.21.15.94:34538) 2024-06-03T00:59:23.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 -- 172.21.15.94:0/3946236728 learned_addr learned my addr 172.21.15.94:0/3946236728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96ca00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 0x7fa96819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 -- 172.21.15.94:0/3946236728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 msgr2=0x7fa96819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 0x7fa96819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 -- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa968074ef0 msgr2=0x7fa96819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa968074ef0 0x7fa96819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.693+0000 7fa96de00700 1 -- 172.21.15.94:0/3946236728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9681a69e0 con 0x7fa96810eae0 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.694+0000 7fa96de00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa96810eae0 0x7fa9681a6310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa96000e940 tx=0x7fa96000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.694+0000 7fa966000700 1 -- 172.21.15.94:0/3946236728 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960004a30 con 0x7fa96810eae0 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.694+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9681a6cd0 con 0x7fa96810eae0 2024-06-03T00:59:23.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.694+0000 7fa966000700 1 -- 172.21.15.94:0/3946236728 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa960005500 con 0x7fa96810eae0 2024-06-03T00:59:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.694+0000 7fa966000700 1 -- 172.21.15.94:0/3946236728 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96000ce20 con 0x7fa96810eae0 2024-06-03T00:59:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.694+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681a7190 con 0x7fa96810eae0 2024-06-03T00:59:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.695+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa968069180 con 0x7fa96810eae0 2024-06-03T00:59:23.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.697+0000 7fa966000700 1 -- 172.21.15.94:0/3946236728 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa960005020 con 0x7fa96810eae0 2024-06-03T00:59:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.698+0000 7fa966000700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa9500a2d20 0x7fa9500a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:23.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.698+0000 7fa966000700 1 -- 172.21.15.94:0/3946236728 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fa9600cde80 con 0x7fa96810eae0 2024-06-03T00:59:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.699+0000 7fa96d400700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa9500a2d20 0x7fa9500a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:59:23.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.699+0000 7fa96d400700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa9500a2d20 0x7fa9500a51e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa95c007f00 tx=0x7fa95c00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:59:23.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.702+0000 7fa966000700 1 -- 172.21.15.94:0/3946236728 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa96009c110 con 0x7fa96810eae0 2024-06-03T00:59:23.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:23.935+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa9680628d0 con 0x7fa9500a2d20 2024-06-03T00:59:23.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:23 smithi094 ceph-mon[32119]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T00:59:23.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:23 smithi094 ceph-mon[32119]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T00:59:23.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:23.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:24.033 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45181]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T00:59:24.033 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:24.033 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45181]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:24.033 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 00:59:23 smithi107 sudo[45181]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:24.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:23 smithi107 ceph-mon[36622]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T00:59:24.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:23 smithi107 ceph-mon[36622]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T00:59:24.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:24.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:24.106 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:23 smithi179 sudo[45936]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T00:59:24.106 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:23 smithi179 sudo[45936]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:24.107 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:23 smithi179 sudo[45936]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:24.107 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:23 smithi179 sudo[45936]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:24.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:23 smithi179 ceph-mon[37041]: osdmap e21: 3 total, 3 up, 3 in 2024-06-03T00:59:24.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:23 smithi179 ceph-mon[37041]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-03T00:59:24.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:24.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:24.605 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:24 smithi179 sudo[45939]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T00:59:24.605 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:24 smithi179 sudo[45939]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:24.605 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:24 smithi179 sudo[45939]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:24.605 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 00:59:24 smithi179 sudo[45939]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:25.035 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:24 smithi094 sudo[51695]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T00:59:25.035 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:24 smithi094 sudo[51695]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:25.035 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:24 smithi094 sudo[51695]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:25.035 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:24 smithi094 sudo[51695]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 sudo[51718]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 sudo[51718]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 sudo[51718]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 sudo[51718]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:25.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:25 smithi094 ceph-mon[32119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:25.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:59:25.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:59:25.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:25.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:25.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:25.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45942]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T00:59:25.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45942]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:25.610 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45942]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:25.610 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45942]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:25.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:25 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:59:25.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:25 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:59:25.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:25 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:25.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:25 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:25.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:25 smithi107 ceph-mon[36622]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45945]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T00:59:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45945]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45945]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:25 smithi179 sudo[45945]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:26.346 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-03T00:59:26.346 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:26.346 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45184]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:26.346 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45184]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:26.622 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:59:26.622 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:26.622 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.622 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.622 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:26.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:26 smithi094 ceph-mon[32119]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-03T00:59:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-03T00:59:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 sudo[45187]: pam_unix(sudo:session): session closed for user root 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:26.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:26.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:26 smithi107 ceph-mon[36622]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:59:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T00:59:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:26 smithi179 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-03T00:59:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:27 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3546388121' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43ade6dd-20a3-419d-8dba-e718f61a022e"}]: dispatch 2024-06-03T00:59:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:27 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3546388121' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43ade6dd-20a3-419d-8dba-e718f61a022e"}]': finished 2024-06-03T00:59:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:27 smithi107 ceph-mon[36622]: osdmap e22: 4 total, 3 up, 4 in 2024-06-03T00:59:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:27 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3546388121' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43ade6dd-20a3-419d-8dba-e718f61a022e"}]: dispatch 2024-06-03T00:59:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:27 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3546388121' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43ade6dd-20a3-419d-8dba-e718f61a022e"}]': finished 2024-06-03T00:59:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:27 smithi094 ceph-mon[32119]: osdmap e22: 4 total, 3 up, 4 in 2024-06-03T00:59:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:27 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3546388121' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43ade6dd-20a3-419d-8dba-e718f61a022e"}]: dispatch 2024-06-03T00:59:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:27 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3546388121' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43ade6dd-20a3-419d-8dba-e718f61a022e"}]': finished 2024-06-03T00:59:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:27 smithi179 ceph-mon[37041]: osdmap e22: 4 total, 3 up, 4 in 2024-06-03T00:59:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:29.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:28 smithi107 ceph-mon[36622]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:29.294 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:28 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3890647217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:59:29.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:28 smithi094 ceph-mon[32119]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:29.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:28 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3890647217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:59:29.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:28 smithi179 ceph-mon[37041]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:29.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:28 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3890647217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T00:59:31.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:31 smithi107 ceph-mon[36622]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:31 smithi094 ceph-mon[32119]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:31 smithi179 ceph-mon[37041]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:33.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:33 smithi094 ceph-mon[32119]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 62 KiB/s wr, 5 op/s 2024-06-03T00:59:33.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:33 smithi107 ceph-mon[36622]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 62 KiB/s wr, 5 op/s 2024-06-03T00:59:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:33 smithi179 ceph-mon[37041]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 62 KiB/s wr, 5 op/s 2024-06-03T00:59:35.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:35 smithi094 ceph-mon[32119]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2024-06-03T00:59:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:35 smithi107 ceph-mon[36622]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2024-06-03T00:59:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:35 smithi179 ceph-mon[37041]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 42 KiB/s wr, 4 op/s 2024-06-03T00:59:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:37 smithi107 ceph-mon[36622]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2024-06-03T00:59:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T00:59:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:37 smithi094 ceph-mon[32119]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2024-06-03T00:59:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T00:59:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:37 smithi179 ceph-mon[37041]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 32 KiB/s wr, 3 op/s 2024-06-03T00:59:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-03T00:59:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:38.531 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:38 smithi094 ceph-mon[32119]: Deploying daemon osd.3 on smithi094 2024-06-03T00:59:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:38 smithi107 ceph-mon[36622]: Deploying daemon osd.3 on smithi094 2024-06-03T00:59:38.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:38 smithi179 ceph-mon[37041]: Deploying daemon osd.3 on smithi094 2024-06-03T00:59:39.334 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:39 smithi094 ceph-mon[32119]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 999 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-03T00:59:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:39 smithi107 ceph-mon[36622]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 999 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-03T00:59:39.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:39 smithi179 ceph-mon[37041]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 999 B/s rd, 26 KiB/s wr, 2 op/s 2024-06-03T00:59:41.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:41 smithi107 ceph-mon[36622]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-06-03T00:59:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:41 smithi094 ceph-mon[32119]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-06-03T00:59:41.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:41 smithi179 ceph-mon[37041]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-06-03T00:59:43.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:43 smithi107 ceph-mon[36622]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-06-03T00:59:43.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:43 smithi094 ceph-mon[32119]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-06-03T00:59:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:43 smithi179 ceph-mon[37041]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-06-03T00:59:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:45 smithi107 ceph-mon[36622]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:45 smithi094 ceph-mon[32119]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:45 smithi179 ceph-mon[37041]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:46.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:46.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:46.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:46.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:46 smithi094 ceph-mon[32119]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:46.763 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:46.763 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:46.763 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:46.764 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:46 smithi179 ceph-mon[37041]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:46 smithi107 ceph-mon[36622]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:49.481 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:49 smithi094 ceph-mon[32119]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:49.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:49.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:49.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:49 smithi107 ceph-mon[36622]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:49.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:49.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:49.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:49 smithi179 ceph-mon[37041]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:50.321 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 3 on host 'smithi094' 2024-06-03T00:59:50.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.320+0000 7fa966000700 1 -- 172.21.15.94:0/3946236728 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa9680628d0 con 0x7fa9500a2d20 2024-06-03T00:59:50.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.324+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa9500a2d20 msgr2=0x7fa9500a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.324+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa9500a2d20 0x7fa9500a51e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa95c007f00 tx=0x7fa95c00b700 comp rx=0 tx=0).stop 2024-06-03T00:59:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.324+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa96810eae0 msgr2=0x7fa9681a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.324+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa96810eae0 0x7fa9681a6310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa96000e940 tx=0x7fa96000ec50 comp rx=0 tx=0).stop 2024-06-03T00:59:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 shutdown_connections 2024-06-03T00:59:50.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa96810eae0 0x7fa9681a6310 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa9500a2d20 0x7fa9500a51e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa968074ef0 0x7fa96819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 --2- 172.21.15.94:0/3946236728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9680764c0 0x7fa96819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 >> 172.21.15.94:0/3946236728 conn(0x7fa9680fda60 msgr2=0x7fa968101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:59:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 shutdown_connections 2024-06-03T00:59:50.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T00:59:50.325+0000 7fa96fe00700 1 -- 172.21.15.94:0/3946236728 wait complete. 2024-06-03T00:59:50.344 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:50 smithi094 ceph-mon[32119]: from='osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T00:59:50.345 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:50 smithi094 ceph-mon[32119]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T00:59:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:50 smithi107 ceph-mon[36622]: from='osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T00:59:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:50 smithi107 ceph-mon[36622]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T00:59:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:50 smithi179 ceph-mon[37041]: from='osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T00:59:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:50 smithi179 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-03T00:59:51.180 DEBUG:teuthology.orchestra.run.smithi094:osd.3> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.3.service 2024-06-03T00:59:51.185 INFO:tasks.cephadm:Deploying osd.4 on smithi107 with /dev/vg_nvme/lv_3... 2024-06-03T00:59:51.186 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T00:59:51.429 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: from='osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:51.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:51.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T00:59:51.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T00:59:51.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: from='osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:59:51.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:51.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:59:51.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:51.551 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:51.551 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-03T00:59:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: osdmap e23: 4 total, 3 up, 4 in 2024-06-03T00:59:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: from='osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:59:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T00:59:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T00:59:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:52.244 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T00:59:51.958+0000 7f453f200700 -1 osd.3 0 waiting for initial osdmap 2024-06-03T00:59:52.244 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 00:59:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T00:59:51.976+0000 7f4534600700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T00:59:52.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:52 smithi094 ceph-mon[32119]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T00:59:52.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:52 smithi094 ceph-mon[32119]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T00:59:52.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:52.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:52.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:52 smithi094 ceph-mon[32119]: from='osd.3 ' entity='osd.3' 2024-06-03T00:59:52.518 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:52 smithi094 ceph-mon[32119]: osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] boot 2024-06-03T00:59:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:52 smithi107 ceph-mon[36622]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T00:59:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:52 smithi107 ceph-mon[36622]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T00:59:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:52 smithi107 ceph-mon[36622]: from='osd.3 ' entity='osd.3' 2024-06-03T00:59:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:52 smithi107 ceph-mon[36622]: osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] boot 2024-06-03T00:59:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:52 smithi179 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T00:59:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:52 smithi179 ceph-mon[37041]: osdmap e24: 4 total, 3 up, 4 in 2024-06-03T00:59:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:52 smithi179 ceph-mon[37041]: from='osd.3 ' entity='osd.3' 2024-06-03T00:59:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:52 smithi179 ceph-mon[37041]: osd.3 [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] boot 2024-06-03T00:59:53.354 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T00:59:53.354 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T00:59:53.354 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-03T00:59:53.354 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-03T00:59:53.354 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012616 s, 831 MB/s 2024-06-03T00:59:53.354 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-03T00:59:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:53 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T00:59:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:53 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T00:59:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:53 smithi107 ceph-mon[36622]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:53 smithi107 ceph-mon[36622]: osdmap e25: 4 total, 4 up, 4 in 2024-06-03T00:59:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:53 smithi107 ceph-mon[36622]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T00:59:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:53 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T00:59:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:53 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T00:59:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:53 smithi094 ceph-mon[32119]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:53 smithi094 ceph-mon[32119]: osdmap e25: 4 total, 4 up, 4 in 2024-06-03T00:59:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:53 smithi094 ceph-mon[32119]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T00:59:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:53 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T00:59:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:53 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T00:59:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:53 smithi179 ceph-mon[37041]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-06-03T00:59:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:53 smithi179 ceph-mon[37041]: osdmap e25: 4 total, 4 up, 4 in 2024-06-03T00:59:53.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T00:59:53.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:53 smithi179 ceph-mon[37041]: osdmap e26: 4 total, 4 up, 4 in 2024-06-03T00:59:53.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-06-03T00:59:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T00:59:55.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:54 smithi107 ceph-mon[36622]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T00:59:55.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:54 smithi107 ceph-mon[36622]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 357 GiB / 358 GiB avail 2024-06-03T00:59:55.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:54 smithi094 ceph-mon[32119]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T00:59:55.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:54 smithi094 ceph-mon[32119]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 357 GiB / 358 GiB avail 2024-06-03T00:59:55.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:54 smithi179 ceph-mon[37041]: osdmap e27: 4 total, 4 up, 4 in 2024-06-03T00:59:55.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:54 smithi179 ceph-mon[37041]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 357 GiB / 358 GiB avail 2024-06-03T00:59:55.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 -- 172.21.15.107:0/1112338512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 msgr2=0x7f053c105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 --2- 172.21.15.107:0/1112338512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c105430 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f052c009990 tx=0x7f052c009ca0 comp rx=0 tx=0).stop 2024-06-03T00:59:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 -- 172.21.15.107:0/1112338512 shutdown_connections 2024-06-03T00:59:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 --2- 172.21.15.107:0/1112338512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c105430 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 --2- 172.21.15.107:0/1112338512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f053c1082a0 0x7f053c10a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 --2- 172.21.15.107:0/1112338512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f053c105970 0x7f053c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 -- 172.21.15.107:0/1112338512 >> 172.21.15.107:0/1112338512 conn(0x7f053c0fc990 msgr2=0x7f053c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 -- 172.21.15.107:0/1112338512 shutdown_connections 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 -- 172.21.15.107:0/1112338512 wait complete. 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 Processor -- start 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.871+0000 7f0541abc700 1 -- start start 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f053c105970 0x7f053c100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f053c1082a0 0x7f053c101530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f053c10e130 con 0x7f053c105970 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f053c10dfb0 con 0x7f053c103040 2024-06-03T00:59:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f053c10e2b0 con 0x7f053c1082a0 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c1029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36880/0 (socket says 172.21.15.107:36880) 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 -- 172.21.15.107:0/464010875 learned_addr learned my addr 172.21.15.107:0/464010875 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053bfff700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f053c1082a0 0x7f053c101530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 -- 172.21.15.107:0/464010875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f053c1082a0 msgr2=0x7f053c101530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f053c1082a0 0x7f053c101530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 -- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f053c105970 msgr2=0x7f053c100ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f053c105970 0x7f053c100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 -- 172.21.15.107:0/464010875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c101de0 con 0x7f053c103040 2024-06-03T00:59:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f053b7fe700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c1029a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f052c005910 tx=0x7f052c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f05397fa700 1 -- 172.21.15.107:0/464010875 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c027070 con 0x7f053c103040 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f05397fa700 1 -- 172.21.15.107:0/464010875 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052c004210 con 0x7f053c103040 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c1a2830 con 0x7f053c103040 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f05397fa700 1 -- 172.21.15.107:0/464010875 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c00f460 con 0x7f053c103040 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.872+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c1a2d00 con 0x7f053c103040 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.873+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c069180 con 0x7f053c103040 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.874+0000 7f05397fa700 1 -- 172.21.15.107:0/464010875 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f052c02a070 con 0x7f053c103040 2024-06-03T00:59:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.875+0000 7f05397fa700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f05280a2ca0 0x7f05280a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T00:59:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.875+0000 7f05397fa700 1 -- 172.21.15.107:0/464010875 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f052c0cf480 con 0x7f053c103040 2024-06-03T00:59:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.875+0000 7f0533fff700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f05280a2ca0 0x7f05280a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T00:59:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.876+0000 7f0533fff700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f05280a2ca0 0x7f05280a5160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0524005950 tx=0x7f05240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T00:59:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:55.880+0000 7f05397fa700 1 -- 172.21.15.107:0/464010875 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f052c031070 con 0x7f053c103040 2024-06-03T00:59:56.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T00:59:56.083+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f053c0628d0 con 0x7f05280a2ca0 2024-06-03T00:59:56.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:55 smithi107 ceph-mon[36622]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T00:59:56.342 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:55 smithi094 ceph-mon[32119]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T00:59:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:55 smithi179 ceph-mon[37041]: osdmap e28: 4 total, 4 up, 4 in 2024-06-03T00:59:57.024 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:56 smithi107 ceph-mon[36622]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T00:59:57.024 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:56 smithi107 ceph-mon[36622]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:59:57.024 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:59:57.024 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:59:57.024 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:57.024 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:56 smithi107 ceph-mon[36622]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 357 GiB / 358 GiB avail 2024-06-03T00:59:57.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:56 smithi094 ceph-mon[32119]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T00:59:57.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:56 smithi094 ceph-mon[32119]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:59:57.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:59:57.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:59:57.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:57.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:56 smithi094 ceph-mon[32119]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 357 GiB / 358 GiB avail 2024-06-03T00:59:57.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:56 smithi179 ceph-mon[37041]: osdmap e29: 4 total, 4 up, 4 in 2024-06-03T00:59:57.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:56 smithi179 ceph-mon[37041]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T00:59:57.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T00:59:57.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T00:59:57.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:57.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:56 smithi179 ceph-mon[37041]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 357 GiB / 358 GiB avail 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi094 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi094 to 8544M 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:58.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:58.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:58 smithi107 ceph-mon[36622]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-03T00:59:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi094 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi094 to 8544M 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:58 smithi094 ceph-mon[32119]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-03T00:59:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi094 2024-06-03T00:59:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T00:59:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi094 to 8544M 2024-06-03T00:59:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T00:59:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T00:59:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T00:59:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:58 smithi179 ceph-mon[37041]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-03T00:59:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:59 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/1092967448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]: dispatch 2024-06-03T00:59:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:59 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]: dispatch 2024-06-03T00:59:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:59 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]': finished 2024-06-03T00:59:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:59 smithi107 ceph-mon[36622]: osdmap e30: 5 total, 4 up, 5 in 2024-06-03T00:59:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 00:59:59 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T00:59:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:59 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/1092967448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]: dispatch 2024-06-03T00:59:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:59 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]: dispatch 2024-06-03T00:59:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:59 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]': finished 2024-06-03T00:59:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:59 smithi094 ceph-mon[32119]: osdmap e30: 5 total, 4 up, 5 in 2024-06-03T00:59:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 00:59:59 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T00:59:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:59 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/1092967448' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]: dispatch 2024-06-03T00:59:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:59 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]: dispatch 2024-06-03T00:59:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:59 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ed0339d-4210-4398-9092-db7bcba4048b"}]': finished 2024-06-03T00:59:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:59 smithi179 ceph-mon[37041]: osdmap e30: 5 total, 4 up, 5 in 2024-06-03T00:59:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 00:59:59 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:00 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/365125623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:00 smithi107 ceph-mon[36622]: overall HEALTH_OK 2024-06-03T01:00:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:00 smithi107 ceph-mon[36622]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:00:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:00 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/365125623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:00 smithi094 ceph-mon[32119]: overall HEALTH_OK 2024-06-03T01:00:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:00 smithi094 ceph-mon[32119]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:00:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:00 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/365125623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:00 smithi179 ceph-mon[37041]: overall HEALTH_OK 2024-06-03T01:00:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:00 smithi179 ceph-mon[37041]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:00:03.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:03 smithi094 ceph-mon[32119]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-06-03T01:00:03.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:03 smithi107 ceph-mon[36622]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-06-03T01:00:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:03 smithi179 ceph-mon[37041]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-06-03T01:00:05.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:05 smithi107 ceph-mon[36622]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T01:00:05.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:05 smithi094 ceph-mon[32119]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T01:00:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:05 smithi179 ceph-mon[37041]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-06-03T01:00:06.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:06 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:00:06.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:06 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:06 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:00:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:06 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:06 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-03T01:00:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:06 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:07.501 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:07 smithi094 ceph-mon[32119]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T01:00:07.501 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:07 smithi094 ceph-mon[32119]: Deploying daemon osd.4 on smithi107 2024-06-03T01:00:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:07 smithi107 ceph-mon[36622]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T01:00:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:07 smithi107 ceph-mon[36622]: Deploying daemon osd.4 on smithi107 2024-06-03T01:00:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:07 smithi179 ceph-mon[37041]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-03T01:00:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:07 smithi179 ceph-mon[37041]: Deploying daemon osd.4 on smithi107 2024-06-03T01:00:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:09 smithi107 ceph-mon[36622]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:09 smithi094 ceph-mon[32119]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:09 smithi179 ceph-mon[37041]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:11.495 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:11 smithi107 ceph-mon[36622]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:11 smithi094 ceph-mon[32119]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:11 smithi179 ceph-mon[37041]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:13.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:13.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:13.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:13.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:12 smithi107 ceph-mon[36622]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:13.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:13.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:13.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:13.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:12 smithi094 ceph-mon[32119]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:13.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:13.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:13.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:13.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:12 smithi179 ceph-mon[37041]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:15.438 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:15 smithi107 ceph-mon[36622]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:15.439 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.439 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.439 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:15.439 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:15.439 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.520 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 4 on host 'smithi107' 2024-06-03T01:00:15.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.521+0000 7f05397fa700 1 -- 172.21.15.107:0/464010875 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f053c0628d0 con 0x7f05280a2ca0 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f05280a2ca0 msgr2=0x7f05280a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f05280a2ca0 0x7f05280a5160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0524005950 tx=0x7f05240058e0 comp rx=0 tx=0).stop 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 msgr2=0x7f053c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c1029a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f052c005910 tx=0x7f052c009580 comp rx=0 tx=0).stop 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 shutdown_connections 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c103040 0x7f053c1029a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f05280a2ca0 0x7f05280a5160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f053c1082a0 0x7f053c101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 --2- 172.21.15.107:0/464010875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f053c105970 0x7f053c100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 >> 172.21.15.107:0/464010875 conn(0x7f053c0fc990 msgr2=0x7f053c1039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:00:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 shutdown_connections 2024-06-03T01:00:15.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:00:15.523+0000 7f0541abc700 1 -- 172.21.15.107:0/464010875 wait complete. 2024-06-03T01:00:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:15 smithi094 ceph-mon[32119]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:15 smithi179 ceph-mon[37041]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:16.207 DEBUG:teuthology.orchestra.run.smithi107:osd.4> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.4.service 2024-06-03T01:00:16.209 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-06-03T01:00:16.210 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-03T01:00:16.451 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:16.452 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:16.452 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:16.452 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:16 smithi107 ceph-mon[36622]: from='osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:00:16.452 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:16 smithi107 ceph-mon[36622]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:00:16.458 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:00:16.507 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:16.507 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:16.507 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:16.507 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:16 smithi179 ceph-mon[37041]: from='osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:00:16.507 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:16 smithi179 ceph-mon[37041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:00:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:16 smithi094 ceph-mon[32119]: from='osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:00:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:16 smithi094 ceph-mon[32119]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-03T01:00:17.524 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:17 smithi107 ceph-mon[36622]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:17.525 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:17 smithi107 ceph-mon[36622]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T01:00:17.525 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:17 smithi107 ceph-mon[36622]: from='osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:00:17.525 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:17 smithi107 ceph-mon[36622]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T01:00:17.525 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:17 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:17.525 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:17 smithi107 ceph-mon[36622]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:00:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:17 smithi094 ceph-mon[32119]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:17 smithi094 ceph-mon[32119]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T01:00:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:17 smithi094 ceph-mon[32119]: from='osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:00:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:17 smithi094 ceph-mon[32119]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T01:00:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:17 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:17 smithi094 ceph-mon[32119]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:00:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:17 smithi179 ceph-mon[37041]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:17 smithi179 ceph-mon[37041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-03T01:00:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:17 smithi179 ceph-mon[37041]: from='osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:00:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:17 smithi179 ceph-mon[37041]: osdmap e31: 5 total, 4 up, 5 in 2024-06-03T01:00:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:17 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:17 smithi179 ceph-mon[37041]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:00:18.509 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4[47975]: 2024-06-03T01:00:18.202+0000 7fde42e2e700 -1 osd.4 0 waiting for initial osdmap 2024-06-03T01:00:18.509 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4[47975]: 2024-06-03T01:00:18.208+0000 7fde3ac19700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:00:18.558 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-03T01:00:18.558 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-03T01:00:18.558 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-03T01:00:18.558 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-03T01:00:18.558 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114983 s, 912 MB/s 2024-06-03T01:00:18.558 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-03T01:00:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-mon[36622]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:00:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-mon[36622]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T01:00:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-mon[36622]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:18 smithi107 ceph-mon[36622]: from='osd.4 ' entity='osd.4' 2024-06-03T01:00:18.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:18 smithi094 ceph-mon[32119]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:00:18.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:18 smithi094 ceph-mon[32119]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T01:00:18.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:18 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:18 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:18 smithi094 ceph-mon[32119]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:18 smithi094 ceph-mon[32119]: from='osd.4 ' entity='osd.4' 2024-06-03T01:00:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:18 smithi179 ceph-mon[37041]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:00:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:18 smithi179 ceph-mon[37041]: osdmap e32: 5 total, 4 up, 5 in 2024-06-03T01:00:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:18 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:18 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:18 smithi179 ceph-mon[37041]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-06-03T01:00:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:18 smithi179 ceph-mon[37041]: from='osd.4 ' entity='osd.4' 2024-06-03T01:00:19.221 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-06-03T01:00:19.459 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:00:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:00:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:00:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] boot 2024-06-03T01:00:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: osdmap e33: 5 total, 5 up, 5 in 2024-06-03T01:00:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:19.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:19.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:19.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:19.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] boot 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: osdmap e33: 5 total, 5 up, 5 in 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.792 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:19.793 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:19.793 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.793 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:19.793 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:19.793 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: osd.4 [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] boot 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: osdmap e33: 5 total, 5 up, 5 in 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:19.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:20.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:20 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi107 2024-06-03T01:00:20.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:20 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi107 to 8544M 2024-06-03T01:00:20.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:20 smithi107 ceph-mon[36622]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:20 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi107 2024-06-03T01:00:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:20 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi107 to 8544M 2024-06-03T01:00:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:20 smithi094 ceph-mon[32119]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:20 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi107 2024-06-03T01:00:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:20 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi107 to 8544M 2024-06-03T01:00:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:20 smithi179 ceph-mon[37041]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.076+0000 7f1124878700 1 -- 172.21.15.179:0/208212186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11201020b0 msgr2=0x7f1120109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.076+0000 7f1124878700 1 --2- 172.21.15.179:0/208212186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11201020b0 0x7f1120109970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f111800b0d0 tx=0x7f111800b3e0 comp rx=0 tx=0).stop 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.077+0000 7f1124878700 1 -- 172.21.15.179:0/208212186 shutdown_connections 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.077+0000 7f1124878700 1 --2- 172.21.15.179:0/208212186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11201029d0 0x7f1120102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.077+0000 7f1124878700 1 --2- 172.21.15.179:0/208212186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11201020b0 0x7f1120109970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.077+0000 7f1124878700 1 --2- 172.21.15.179:0/208212186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1120101710 0x7f1120101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.077+0000 7f1124878700 1 -- 172.21.15.179:0/208212186 >> 172.21.15.179:0/208212186 conn(0x7f11200fc160 msgr2=0x7f11200fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.077+0000 7f1124878700 1 -- 172.21.15.179:0/208212186 shutdown_connections 2024-06-03T01:00:21.078 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.077+0000 7f1124878700 1 -- 172.21.15.179:0/208212186 wait complete. 2024-06-03T01:00:21.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 Processor -- start 2024-06-03T01:00:21.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 -- start start 2024-06-03T01:00:21.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1120101710 0x7f11201011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:21.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11201020b0 0x7f112019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:21.080 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11201029d0 0x7f112019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f112010d0a0 con 0x7f11201020b0 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f112010cf20 con 0x7f11201029d0 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f1124878700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f112010d220 con 0x7f1120101710 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1120101710 0x7f11201011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1120101710 0x7f11201011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33490/0 (socket says 172.21.15.179:33490) 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 -- 172.21.15.179:0/138712417 learned_addr learned my addr 172.21.15.179:0/138712417 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 -- 172.21.15.179:0/138712417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11201029d0 msgr2=0x7f112019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11201029d0 0x7f112019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 -- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11201020b0 msgr2=0x7f112019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11201020b0 0x7f112019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 -- 172.21.15.179:0/138712417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11201a03c0 con 0x7f1120101710 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f111e59c700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1120101710 0x7f11201011c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f111000d980 tx=0x7f111000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:00:21.081 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.078+0000 7f110f7fe700 1 -- 172.21.15.179:0/138712417 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110017070 con 0x7f1120101710 2024-06-03T01:00:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.079+0000 7f110f7fe700 1 -- 172.21.15.179:0/138712417 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110009b40 con 0x7f1120101710 2024-06-03T01:00:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.079+0000 7f110f7fe700 1 -- 172.21.15.179:0/138712417 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111000f620 con 0x7f1120101710 2024-06-03T01:00:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.079+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11201a06b0 con 0x7f1120101710 2024-06-03T01:00:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.079+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11201a0ba0 con 0x7f1120101710 2024-06-03T01:00:21.082 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.080+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1120069180 con 0x7f1120101710 2024-06-03T01:00:21.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.081+0000 7f110f7fe700 1 -- 172.21.15.179:0/138712417 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1110010460 con 0x7f1120101710 2024-06-03T01:00:21.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.081+0000 7f110f7fe700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f11080a2c20 0x7f11080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:21.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.081+0000 7f110f7fe700 1 -- 172.21.15.179:0/138712417 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f11100cdc60 con 0x7f1120101710 2024-06-03T01:00:21.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.082+0000 7f111dd9b700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f11080a2c20 0x7f11080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:00:21.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.083+0000 7f111dd9b700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f11080a2c20 0x7f11080a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f11140097b0 tx=0x7f1114006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:00:21.085 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.085+0000 7f110f7fe700 1 -- 172.21.15.179:0/138712417 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f111009bc30 con 0x7f1120101710 2024-06-03T01:00:21.286 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:21.286+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f11200628d0 con 0x7f11080a2c20 2024-06-03T01:00:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:21 smithi179 ceph-mon[37041]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T01:00:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:00:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:00:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:21 smithi107 ceph-mon[36622]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T01:00:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:00:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:00:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:21.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:21 smithi094 ceph-mon[32119]: osdmap e34: 5 total, 5 up, 5 in 2024-06-03T01:00:21.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:00:21.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:00:21.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:22 smithi107 ceph-mon[36622]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:00:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:22 smithi107 ceph-mon[36622]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:22.797 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:22 smithi179 ceph-mon[37041]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:00:22.797 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:22 smithi179 ceph-mon[37041]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:22.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:22 smithi094 ceph-mon[32119]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:00:22.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:22 smithi094 ceph-mon[32119]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:24.274 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:23 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/1493156876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]: dispatch 2024-06-03T01:00:24.274 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:23 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]: dispatch 2024-06-03T01:00:24.275 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:23 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]': finished 2024-06-03T01:00:24.275 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:23 smithi179 ceph-mon[37041]: osdmap e35: 6 total, 5 up, 6 in 2024-06-03T01:00:24.275 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:23 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/1493156876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]: dispatch 2024-06-03T01:00:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:23 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]: dispatch 2024-06-03T01:00:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:23 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]': finished 2024-06-03T01:00:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:23 smithi107 ceph-mon[36622]: osdmap e35: 6 total, 5 up, 6 in 2024-06-03T01:00:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:23 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/1493156876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]: dispatch 2024-06-03T01:00:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:23 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]: dispatch 2024-06-03T01:00:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:23 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46c6631-7947-49fd-afa2-eb39f3145766"}]': finished 2024-06-03T01:00:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:23 smithi094 ceph-mon[32119]: osdmap e35: 6 total, 5 up, 6 in 2024-06-03T01:00:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:25.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:24 smithi107 ceph-mon[36622]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:25.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:24 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/3986141300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:24 smithi094 ceph-mon[32119]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:24 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/3986141300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:25.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:24 smithi179 ceph-mon[37041]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:25.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:24 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/3986141300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:27 smithi107 ceph-mon[36622]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:27.563 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:27 smithi179 ceph-mon[37041]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:27 smithi094 ceph-mon[32119]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:29.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:29 smithi107 ceph-mon[36622]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:29 smithi094 ceph-mon[32119]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:29 smithi179 ceph-mon[37041]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:31.304 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:31 smithi179 ceph-mon[37041]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:31.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:31 smithi107 ceph-mon[36622]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:31 smithi094 ceph-mon[32119]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:32 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:00:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:32 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:32 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:00:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:32 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:32 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-03T01:00:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:32 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:33.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:33 smithi107 ceph-mon[36622]: Deploying daemon osd.5 on smithi179 2024-06-03T01:00:33.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:33 smithi107 ceph-mon[36622]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:33 smithi094 ceph-mon[32119]: Deploying daemon osd.5 on smithi179 2024-06-03T01:00:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:33 smithi094 ceph-mon[32119]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:33 smithi179 ceph-mon[37041]: Deploying daemon osd.5 on smithi179 2024-06-03T01:00:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:33 smithi179 ceph-mon[37041]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:35 smithi107 ceph-mon[36622]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:35 smithi094 ceph-mon[32119]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:35 smithi179 ceph-mon[37041]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:37 smithi107 ceph-mon[36622]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:37 smithi094 ceph-mon[32119]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:37.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:37 smithi179 ceph-mon[37041]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:37.602 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:37.602 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:37.602 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:39 smithi179 ceph-mon[37041]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:39 smithi107 ceph-mon[36622]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:39.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:39 smithi094 ceph-mon[32119]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:40.580 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.580+0000 7f110f7fe700 1 -- 172.21.15.179:0/138712417 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11200628d0 con 0x7f11080a2c20 2024-06-03T01:00:40.580 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-06-03T01:00:40.582 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f11080a2c20 msgr2=0x7f11080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f11080a2c20 0x7f11080a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f11140097b0 tx=0x7f1114006d20 comp rx=0 tx=0).stop 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1120101710 msgr2=0x7f11201011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1120101710 0x7f11201011c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f111000d980 tx=0x7f111000dd40 comp rx=0 tx=0).stop 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 shutdown_connections 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11201029d0 0x7f112019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f11080a2c20 0x7f11080a50e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1120101710 0x7f11201011c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 --2- 172.21.15.179:0/138712417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11201020b0 0x7f112019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 >> 172.21.15.179:0/138712417 conn(0x7f11200fc160 msgr2=0x7f1120105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 shutdown_connections 2024-06-03T01:00:40.583 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:00:40.583+0000 7f1124878700 1 -- 172.21.15.179:0/138712417 wait complete. 2024-06-03T01:00:40.813 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:40.813 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:40.813 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:40.813 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:40.813 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:40.813 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:40.814 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:40.814 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:40.814 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:40 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:41.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:41.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:41.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:41.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:40 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:41.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:41.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:41.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:41.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:41.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:40 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:00:41.274 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.5.service 2024-06-03T01:00:41.277 INFO:tasks.cephadm:Deploying osd.6 on smithi094 with /dev/vg_nvme/lv_2... 2024-06-03T01:00:41.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T01:00:41.757 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:00:41.851 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:41 smithi179 ceph-mon[37041]: from='osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:00:41.851 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:41 smithi179 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:00:42.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:41 smithi107 ceph-mon[36622]: from='osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:00:42.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:41 smithi107 ceph-mon[36622]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:00:42.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:41 smithi094 ceph-mon[32119]: from='osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:00:42.071 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:41 smithi094 ceph-mon[32119]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-03T01:00:42.817 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:42 smithi179 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T01:00:42.817 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:42 smithi179 ceph-mon[37041]: from='osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:00:42.817 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:42 smithi179 ceph-mon[37041]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T01:00:42.817 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:42 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:42.818 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:42 smithi179 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:00:42.818 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:42 smithi179 ceph-mon[37041]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:43.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:42 smithi107 ceph-mon[36622]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T01:00:43.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:42 smithi107 ceph-mon[36622]: from='osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:00:43.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:42 smithi107 ceph-mon[36622]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T01:00:43.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:42 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:43.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:42 smithi107 ceph-mon[36622]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:00:43.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:42 smithi107 ceph-mon[36622]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:43.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:42 smithi094 ceph-mon[32119]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-03T01:00:43.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:42 smithi094 ceph-mon[32119]: from='osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:00:43.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:42 smithi094 ceph-mon[32119]: osdmap e36: 6 total, 5 up, 6 in 2024-06-03T01:00:43.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:42 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:43.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:42 smithi094 ceph-mon[32119]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:00:43.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:42 smithi094 ceph-mon[32119]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:43.768 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:00:43.516+0000 7f5b2237a700 -1 osd.5 0 waiting for initial osdmap 2024-06-03T01:00:43.768 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:00:43.523+0000 7f5b1a165700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:00:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:43 smithi094 ceph-mon[32119]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:00:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:43 smithi094 ceph-mon[32119]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T01:00:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:43 smithi094 ceph-mon[32119]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T01:00:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:43 smithi094 ceph-mon[32119]: from='osd.5 ' entity='osd.5' 2024-06-03T01:00:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:44.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:43 smithi107 ceph-mon[36622]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:00:44.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:43 smithi107 ceph-mon[36622]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T01:00:44.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:44.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:43 smithi107 ceph-mon[36622]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T01:00:44.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:44.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:43 smithi107 ceph-mon[36622]: from='osd.5 ' entity='osd.5' 2024-06-03T01:00:44.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:44.089 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:00:44.089 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-mon[37041]: osdmap e37: 6 total, 5 up, 6 in 2024-06-03T01:00:44.089 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:44.090 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-mon[37041]: osdmap e38: 6 total, 5 up, 6 in 2024-06-03T01:00:44.090 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:44.090 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-mon[37041]: from='osd.5 ' entity='osd.5' 2024-06-03T01:00:44.090 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T01:00:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T01:00:44.750 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-03T01:00:44.750 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-03T01:00:44.751 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0321312 s, 326 MB/s 2024-06-03T01:00:44.751 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-03T01:00:45.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:00:45.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:00:45.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] boot 2024-06-03T01:00:45.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T01:00:45.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:45.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:45.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:45.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:45.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:45.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:45.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:00:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:00:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] boot 2024-06-03T01:00:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T01:00:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:45.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:45.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:45.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:45.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:45.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: osd.5 [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] boot 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: osdmap e39: 6 total, 6 up, 6 in 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:45.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:00:45.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:45.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:00:45.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:00:45.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-06-03T01:00:46.049 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:00:46.220 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:45 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi179 2024-06-03T01:00:46.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:45 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi179 to 8544M 2024-06-03T01:00:46.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:45 smithi094 ceph-mon[32119]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T01:00:46.282 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:45 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi179 2024-06-03T01:00:46.282 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:45 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi179 to 8544M 2024-06-03T01:00:46.282 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:45 smithi179 ceph-mon[37041]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T01:00:46.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:45 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi179 2024-06-03T01:00:46.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:45 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi179 to 8544M 2024-06-03T01:00:46.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:45 smithi107 ceph-mon[36622]: osdmap e40: 6 total, 6 up, 6 in 2024-06-03T01:00:47.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:46 smithi094 ceph-mon[32119]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T01:00:47.230 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:46 smithi094 ceph-mon[32119]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:47.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:46 smithi107 ceph-mon[36622]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T01:00:47.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:46 smithi107 ceph-mon[36622]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:47.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:46 smithi179 ceph-mon[37041]: osdmap e41: 6 total, 6 up, 6 in 2024-06-03T01:00:47.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:46 smithi179 ceph-mon[37041]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 -- 172.21.15.94:0/137577192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa0108b30 msgr2=0x7f9fa0108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/137577192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa0108b30 0x7f9fa0108f50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f900099c0 tx=0x7f9f90009cd0 comp rx=0 tx=0).stop 2024-06-03T01:00:48.265 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:47 smithi094 ceph-mon[32119]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 -- 172.21.15.94:0/137577192 shutdown_connections 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/137577192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa010a6d0 0x7f9fa0110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/137577192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa0108b30 0x7f9fa0108f50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/137577192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fa0109d30 0x7f9fa010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 -- 172.21.15.94:0/137577192 >> 172.21.15.94:0/137577192 conn(0x7f9fa0077b00 msgr2=0x7f9fa0079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.264+0000 7f9fa5e00700 1 -- 172.21.15.94:0/137577192 shutdown_connections 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.265+0000 7f9fa5e00700 1 -- 172.21.15.94:0/137577192 wait complete. 2024-06-03T01:00:48.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 Processor -- start 2024-06-03T01:00:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 -- start start 2024-06-03T01:00:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fa0108b30 0x7f9fa019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa0109d30 0x7f9fa019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa010a6d0 0x7f9fa01a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0114710 con 0x7f9fa010a6d0 2024-06-03T01:00:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0114590 con 0x7f9fa0109d30 2024-06-03T01:00:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.266+0000 7f9fa5e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0114890 con 0x7f9fa0108b30 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa010a6d0 0x7f9fa01a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9ea00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fa0108b30 0x7f9fa019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa010a6d0 0x7f9fa01a62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48624/0 (socket says 172.21.15.94:48624) 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9f400700 1 -- 172.21.15.94:0/1572479079 learned_addr learned my addr 172.21.15.94:0/1572479079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9e000700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa0109d30 0x7f9fa019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9ea00700 1 -- 172.21.15.94:0/1572479079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa0109d30 msgr2=0x7f9fa019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9ea00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa0109d30 0x7f9fa019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9ea00700 1 -- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa010a6d0 msgr2=0x7f9fa01a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9ea00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa010a6d0 0x7f9fa01a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9ea00700 1 -- 172.21.15.94:0/1572479079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa01a6970 con 0x7f9fa0108b30 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.267+0000 7f9f9f400700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa010a6d0 0x7f9fa01a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:00:48.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.268+0000 7f9f9ea00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fa0108b30 0x7f9fa019ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f900098e0 tx=0x7f9f9000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:00:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.268+0000 7f9fa4a00700 1 -- 172.21.15.94:0/1572479079 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90009e60 con 0x7f9fa0108b30 2024-06-03T01:00:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.268+0000 7f9fa4a00700 1 -- 172.21.15.94:0/1572479079 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f90003e30 con 0x7f9fa0108b30 2024-06-03T01:00:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.268+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa01a6c00 con 0x7f9fa0108b30 2024-06-03T01:00:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.268+0000 7f9fa4a00700 1 -- 172.21.15.94:0/1572479079 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000fd90 con 0x7f9fa0108b30 2024-06-03T01:00:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.268+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa01a70e0 con 0x7f9fa0108b30 2024-06-03T01:00:48.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.269+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa0069180 con 0x7f9fa0108b30 2024-06-03T01:00:48.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.271+0000 7f9fa4a00700 1 -- 172.21.15.94:0/1572479079 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9f9000f4f0 con 0x7f9fa0108b30 2024-06-03T01:00:48.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.272+0000 7f9fa4a00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9f840a2c20 0x7f9f840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:00:48.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.272+0000 7f9fa4a00700 1 -- 172.21.15.94:0/1572479079 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f9f900cef20 con 0x7f9fa0108b30 2024-06-03T01:00:48.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.272+0000 7f9f9e000700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9f840a2c20 0x7f9f840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:00:48.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.273+0000 7f9f9e000700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9f840a2c20 0x7f9f840a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009c60 tx=0x7f9f8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:00:48.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.277+0000 7f9fa4a00700 1 -- 172.21.15.94:0/1572479079 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f90013070 con 0x7f9fa0108b30 2024-06-03T01:00:48.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:47 smithi107 ceph-mon[36622]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T01:00:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:47 smithi179 ceph-mon[37041]: osdmap e42: 6 total, 6 up, 6 in 2024-06-03T01:00:48.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:00:48.510+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9fa00628d0 con 0x7f9f840a2c20 2024-06-03T01:00:49.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:49 smithi107 ceph-mon[36622]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T01:00:49.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:49 smithi107 ceph-mon[36622]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:00:49.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:00:49.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:00:49.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:49.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:49 smithi094 ceph-mon[32119]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T01:00:49.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:49 smithi094 ceph-mon[32119]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:00:49.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:00:49.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:00:49.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:49.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:49 smithi179 ceph-mon[37041]: osdmap e43: 6 total, 6 up, 6 in 2024-06-03T01:00:49.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:49 smithi179 ceph-mon[37041]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:00:49.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:00:49.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:00:49.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:00:50.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:50 smithi107 ceph-mon[36622]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:00:50.338 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:50 smithi094 ceph-mon[32119]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:00:50.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:50 smithi179 ceph-mon[37041]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:00:51.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:51 smithi094 ceph-mon[32119]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-03T01:00:51.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:51 smithi107 ceph-mon[36622]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-03T01:00:51.391 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:51 smithi179 ceph-mon[37041]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-06-03T01:00:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:53 smithi107 ceph-mon[36622]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T01:00:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:53 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/940407714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172b5d67-f3cf-48eb-9060-0b421fb38e63"}]: dispatch 2024-06-03T01:00:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:53 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/940407714' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172b5d67-f3cf-48eb-9060-0b421fb38e63"}]': finished 2024-06-03T01:00:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:53 smithi107 ceph-mon[36622]: osdmap e44: 7 total, 6 up, 7 in 2024-06-03T01:00:53.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:53 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:00:53.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:53 smithi094 ceph-mon[32119]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T01:00:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:53 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/940407714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172b5d67-f3cf-48eb-9060-0b421fb38e63"}]: dispatch 2024-06-03T01:00:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:53 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/940407714' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172b5d67-f3cf-48eb-9060-0b421fb38e63"}]': finished 2024-06-03T01:00:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:53 smithi094 ceph-mon[32119]: osdmap e44: 7 total, 6 up, 7 in 2024-06-03T01:00:53.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:53 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:00:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:53 smithi179 ceph-mon[37041]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-06-03T01:00:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:53 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/940407714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172b5d67-f3cf-48eb-9060-0b421fb38e63"}]: dispatch 2024-06-03T01:00:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:53 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/940407714' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172b5d67-f3cf-48eb-9060-0b421fb38e63"}]': finished 2024-06-03T01:00:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:53 smithi179 ceph-mon[37041]: osdmap e44: 7 total, 6 up, 7 in 2024-06-03T01:00:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:53 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:00:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:54 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1615119796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:54 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1615119796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:54 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1615119796' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:00:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:55 smithi107 ceph-mon[36622]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:55 smithi094 ceph-mon[32119]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:55 smithi179 ceph-mon[37041]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:57 smithi107 ceph-mon[36622]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:57 smithi094 ceph-mon[32119]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:57 smithi179 ceph-mon[37041]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:00:59 smithi107 ceph-mon[36622]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:00:59 smithi094 ceph-mon[32119]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:00:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:00:59 smithi179 ceph-mon[37041]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:01.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:01 smithi094 ceph-mon[32119]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:01.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:01 smithi107 ceph-mon[36622]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:01.555 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:01 smithi179 ceph-mon[37041]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:03.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:03 smithi107 ceph-mon[36622]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:03.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:03 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T01:01:03.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:03 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:03.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:03 smithi094 ceph-mon[32119]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:03.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:03 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T01:01:03.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:03 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:03 smithi179 ceph-mon[37041]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:03 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-03T01:01:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:03 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:04 smithi107 ceph-mon[36622]: Deploying daemon osd.6 on smithi094 2024-06-03T01:01:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:04 smithi094 ceph-mon[32119]: Deploying daemon osd.6 on smithi094 2024-06-03T01:01:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:04 smithi179 ceph-mon[37041]: Deploying daemon osd.6 on smithi094 2024-06-03T01:01:05.425 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:05 smithi094 ceph-mon[32119]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:05.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:05 smithi107 ceph-mon[36622]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:05 smithi179 ceph-mon[37041]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:07.497 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:07 smithi094 ceph-mon[32119]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:07 smithi179 ceph-mon[37041]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:07 smithi107 ceph-mon[36622]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:09.369 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:09 smithi094 ceph-mon[32119]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:09 smithi179 ceph-mon[37041]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:09 smithi107 ceph-mon[36622]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:10.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:10 smithi094 ceph-mon[32119]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:10.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:10 smithi179 ceph-mon[37041]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:10 smithi107 ceph-mon[36622]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:11 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:11 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:11 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:11 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:11 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:11 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:11 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:11 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:11 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:12 smithi179 ceph-mon[37041]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:12 smithi107 ceph-mon[36622]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:12.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:12 smithi094 ceph-mon[32119]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:15 smithi107 ceph-mon[36622]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:15 smithi094 ceph-mon[32119]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:15.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:15 smithi179 ceph-mon[37041]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:15.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:15.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:16.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.173+0000 7f9fa4a00700 1 -- 172.21.15.94:0/1572479079 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9fa00628d0 con 0x7f9f840a2c20 2024-06-03T01:01:16.175 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 6 on host 'smithi094' 2024-06-03T01:01:16.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9f840a2c20 msgr2=0x7f9f840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9f840a2c20 0x7f9f840a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009c60 tx=0x7f9f8c009260 comp rx=0 tx=0).stop 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fa0108b30 msgr2=0x7f9fa019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fa0108b30 0x7f9fa019ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9f900098e0 tx=0x7f9f9000bf60 comp rx=0 tx=0).stop 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 shutdown_connections 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9f840a2c20 0x7f9f840a50e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fa010a6d0 0x7f9fa01a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fa0108b30 0x7f9fa019ed20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 --2- 172.21.15.94:0/1572479079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fa0109d30 0x7f9fa019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 >> 172.21.15.94:0/1572479079 conn(0x7f9fa0077b00 msgr2=0x7f9fa010d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 shutdown_connections 2024-06-03T01:01:16.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:01:16.177+0000 7f9fa5e00700 1 -- 172.21.15.94:0/1572479079 wait complete. 2024-06-03T01:01:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:16 smithi094 ceph-mon[32119]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T01:01:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:16.506 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:16 smithi179 ceph-mon[37041]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T01:01:16.507 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:16.507 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:16 smithi107 ceph-mon[36622]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-03T01:01:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:17.185 DEBUG:teuthology.orchestra.run.smithi094:osd.6> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.6.service 2024-06-03T01:01:17.190 INFO:tasks.cephadm:Deploying osd.7 on smithi107 with /dev/vg_nvme/lv_2... 2024-06-03T01:01:17.190 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T01:01:17.359 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:17 smithi094 ceph-mon[32119]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:17.359 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:17 smithi094 ceph-mon[32119]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T01:01:17.359 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:17 smithi094 ceph-mon[32119]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T01:01:17.359 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:17 smithi094 ceph-mon[32119]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:01:17.359 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:17 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:17.433 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:01:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:17 smithi107 ceph-mon[36622]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:17 smithi107 ceph-mon[36622]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T01:01:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:17 smithi107 ceph-mon[36622]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T01:01:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:17 smithi107 ceph-mon[36622]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:01:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:17 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:17 smithi179 ceph-mon[37041]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:17 smithi179 ceph-mon[37041]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-03T01:01:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:17 smithi179 ceph-mon[37041]: osdmap e45: 7 total, 6 up, 7 in 2024-06-03T01:01:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:17 smithi179 ceph-mon[37041]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:01:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:17 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:17.963 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:01:17 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:01:17.874+0000 7f197f800700 -1 osd.6 0 waiting for initial osdmap 2024-06-03T01:01:17.963 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:01:17 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:01:17.909+0000 7f1975600700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:01:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:18 smithi107 ceph-mon[36622]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T01:01:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:18 smithi107 ceph-mon[36622]: osdmap e46: 7 total, 6 up, 7 in 2024-06-03T01:01:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:18 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:18 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:18 smithi107 ceph-mon[36622]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' 2024-06-03T01:01:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:18 smithi094 ceph-mon[32119]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T01:01:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:18 smithi094 ceph-mon[32119]: osdmap e46: 7 total, 6 up, 7 in 2024-06-03T01:01:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:18 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:18 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:18 smithi094 ceph-mon[32119]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' 2024-06-03T01:01:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:18 smithi179 ceph-mon[37041]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T01:01:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:18 smithi179 ceph-mon[37041]: osdmap e46: 7 total, 6 up, 7 in 2024-06-03T01:01:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:18 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:18 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:18 smithi179 ceph-mon[37041]: from='osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469]' entity='osd.6' 2024-06-03T01:01:19.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:19 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:01:19.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:19 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:01:19.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:19 smithi094 ceph-mon[32119]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:19.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:19.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:19 smithi094 ceph-mon[32119]: osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] boot 2024-06-03T01:01:19.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:19 smithi094 ceph-mon[32119]: osdmap e47: 7 total, 7 up, 7 in 2024-06-03T01:01:19.398 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:19.426 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T01:01:19.426 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T01:01:19.426 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-03T01:01:19.426 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-03T01:01:19.427 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124413 s, 843 MB/s 2024-06-03T01:01:19.427 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-03T01:01:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:19 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:01:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:19 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:01:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:19 smithi107 ceph-mon[36622]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:19 smithi107 ceph-mon[36622]: osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] boot 2024-06-03T01:01:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:19 smithi107 ceph-mon[36622]: osdmap e47: 7 total, 7 up, 7 in 2024-06-03T01:01:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:19 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:01:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:19 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:01:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:19 smithi179 ceph-mon[37041]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-06-03T01:01:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:19 smithi179 ceph-mon[37041]: osd.6 [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] boot 2024-06-03T01:01:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:19 smithi179 ceph-mon[37041]: osdmap e47: 7 total, 7 up, 7 in 2024-06-03T01:01:19.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:01:20.081 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-06-03T01:01:20.327 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:01:21.427 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:21 smithi107 ceph-mon[36622]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:21.428 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:21 smithi107 ceph-mon[36622]: osdmap e48: 7 total, 7 up, 7 in 2024-06-03T01:01:21.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:21 smithi094 ceph-mon[32119]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:21.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:21 smithi094 ceph-mon[32119]: osdmap e48: 7 total, 7 up, 7 in 2024-06-03T01:01:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:21 smithi179 ceph-mon[37041]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:21 smithi179 ceph-mon[37041]: osdmap e48: 7 total, 7 up, 7 in 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 -- 172.21.15.107:0/2426424863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681058b0 msgr2=0x7f3468107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 --2- 172.21.15.107:0/2426424863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681058b0 0x7f3468107ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3464009990 tx=0x7f3464009ca0 comp rx=0 tx=0).stop 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 -- 172.21.15.107:0/2426424863 shutdown_connections 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 --2- 172.21.15.107:0/2426424863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34681081e0 0x7f346810a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 --2- 172.21.15.107:0/2426424863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3468102f80 0x7f3468105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 --2- 172.21.15.107:0/2426424863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681058b0 0x7f3468107ca0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 -- 172.21.15.107:0/2426424863 >> 172.21.15.107:0/2426424863 conn(0x7f34680fc990 msgr2=0x7f34680fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:01:21.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 -- 172.21.15.107:0/2426424863 shutdown_connections 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.975+0000 7f346f3ac700 1 -- 172.21.15.107:0/2426424863 wait complete. 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 Processor -- start 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 -- start start 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3468102f80 0x7f3468198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34681058b0 0x7f3468198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681081e0 0x7f346819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f346810e1c0 con 0x7f3468102f80 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f346810e040 con 0x7f34681081e0 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346f3ac700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f346810e340 con 0x7f34681058b0 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346d949700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681081e0 0x7f346819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346d949700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681081e0 0x7f346819fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51776/0 (socket says 172.21.15.107:51776) 2024-06-03T01:01:21.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.976+0000 7f346d949700 1 -- 172.21.15.107:0/3936554728 learned_addr learned my addr 172.21.15.107:0/3936554728 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346c947700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34681058b0 0x7f3468198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346d949700 1 -- 172.21.15.107:0/3936554728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34681058b0 msgr2=0x7f3468198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346d949700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34681058b0 0x7f3468198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346d949700 1 -- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3468102f80 msgr2=0x7f3468198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346d148700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3468102f80 0x7f3468198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346d949700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3468102f80 0x7f3468198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346d949700 1 -- 172.21.15.107:0/3936554728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34681a04f0 con 0x7f34681081e0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346c947700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34681058b0 0x7f3468198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346d949700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681081e0 0x7f346819fe50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f346000b9e0 tx=0x7f346000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f345e7fc700 1 -- 172.21.15.107:0/3936554728 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460019070 con 0x7f34681081e0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f345e7fc700 1 -- 172.21.15.107:0/3936554728 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460014440 con 0x7f34681081e0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34681a07e0 con 0x7f34681081e0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f345e7fc700 1 -- 172.21.15.107:0/3936554728 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600125a0 con 0x7f34681081e0 2024-06-03T01:01:21.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.977+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34681a0d50 con 0x7f34681081e0 2024-06-03T01:01:21.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.978+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3468069180 con 0x7f34681081e0 2024-06-03T01:01:21.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.980+0000 7f345e7fc700 1 -- 172.21.15.107:0/3936554728 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f346001c070 con 0x7f34681081e0 2024-06-03T01:01:21.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.980+0000 7f345e7fc700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34540a2c20 0x7f34540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:21.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.980+0000 7f345e7fc700 1 -- 172.21.15.107:0/3936554728 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f34600cdbe0 con 0x7f34681081e0 2024-06-03T01:01:21.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.981+0000 7f346d148700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34540a2c20 0x7f34540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:01:21.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.982+0000 7f346d148700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34540a2c20 0x7f34540a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3458006fd0 tx=0x7f3458008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:01:21.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:21.984+0000 7f345e7fc700 1 -- 172.21.15.107:0/3936554728 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f346009b850 con 0x7f34681081e0 2024-06-03T01:01:22.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:22.188+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f34680628d0 con 0x7f34540a2c20 2024-06-03T01:01:22.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:22 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:01:22.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:22 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:01:22.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:22 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:22.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:22 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:22 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:01:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:22 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:01:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:22 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:22 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:22 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:01:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:22 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:01:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:22 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:22 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:01:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi094 2024-06-03T01:01:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:23.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:23 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi094 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:23.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:23.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:23.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:23.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:23 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi094 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:23 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:24.379 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:24 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi094 to 5696M 2024-06-03T01:01:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:24 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi094 to 5696M 2024-06-03T01:01:24.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:24 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi094 to 5696M 2024-06-03T01:01:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:25 smithi094 ceph-mon[32119]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:25 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/237281410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]: dispatch 2024-06-03T01:01:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:25 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]: dispatch 2024-06-03T01:01:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:25 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]': finished 2024-06-03T01:01:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:25 smithi094 ceph-mon[32119]: osdmap e49: 8 total, 7 up, 8 in 2024-06-03T01:01:25.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:25 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:25 smithi179 ceph-mon[37041]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:25 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/237281410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]: dispatch 2024-06-03T01:01:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:25 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]: dispatch 2024-06-03T01:01:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:25 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]': finished 2024-06-03T01:01:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:25 smithi179 ceph-mon[37041]: osdmap e49: 8 total, 7 up, 8 in 2024-06-03T01:01:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:25 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:25.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:25 smithi107 ceph-mon[36622]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:25.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:25 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/237281410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]: dispatch 2024-06-03T01:01:25.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:25 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]: dispatch 2024-06-03T01:01:25.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:25 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05b89eaa-5143-4517-949e-e25d75b4c5b8"}]': finished 2024-06-03T01:01:25.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:25 smithi107 ceph-mon[36622]: osdmap e49: 8 total, 7 up, 8 in 2024-06-03T01:01:25.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:25 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:26 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/1283573138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:01:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:26 smithi094 ceph-mon[32119]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:26 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/1283573138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:01:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:26 smithi179 ceph-mon[37041]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:26.693 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:26 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/1283573138' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:01:26.693 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:26 smithi107 ceph-mon[36622]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:29.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:29 smithi107 ceph-mon[36622]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:29 smithi094 ceph-mon[32119]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:29 smithi179 ceph-mon[37041]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:31.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:31 smithi107 ceph-mon[36622]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:31 smithi094 ceph-mon[32119]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:31 smithi179 ceph-mon[37041]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:33.369 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:33 smithi107 ceph-mon[36622]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:33.369 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:33 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T01:01:33.369 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:33 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:33 smithi094 ceph-mon[32119]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:33 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T01:01:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:33 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:33 smithi179 ceph-mon[37041]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:33 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-03T01:01:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:33 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:34.367 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:34 smithi107 ceph-mon[36622]: Deploying daemon osd.7 on smithi107 2024-06-03T01:01:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:34 smithi094 ceph-mon[32119]: Deploying daemon osd.7 on smithi107 2024-06-03T01:01:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:34 smithi179 ceph-mon[37041]: Deploying daemon osd.7 on smithi107 2024-06-03T01:01:35.382 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:35 smithi107 ceph-mon[36622]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:35 smithi094 ceph-mon[32119]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:35 smithi179 ceph-mon[37041]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:37.416 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:37 smithi107 ceph-mon[36622]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:37 smithi094 ceph-mon[32119]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:37 smithi179 ceph-mon[37041]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:39.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:39.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:39.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:39.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:39 smithi094 ceph-mon[32119]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:39 smithi107 ceph-mon[36622]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:39.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:39.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:39.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:39.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:39 smithi179 ceph-mon[37041]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:41.529 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:41 smithi107 ceph-mon[36622]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:41.529 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:41 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.529 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:41 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.529 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:41 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:41.529 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:41 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:41.529 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:41 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:41 smithi094 ceph-mon[32119]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:41 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:41 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:41 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:41 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:41 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:41 smithi179 ceph-mon[37041]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:41.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:41 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:41 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:41 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:41.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:41 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:41.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:41 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:41.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.969+0000 7f345e7fc700 1 -- 172.21.15.107:0/3936554728 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34680628d0 con 0x7f34540a2c20 2024-06-03T01:01:41.970 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 7 on host 'smithi107' 2024-06-03T01:01:41.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34540a2c20 msgr2=0x7f34540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34540a2c20 0x7f34540a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3458006fd0 tx=0x7f3458008040 comp rx=0 tx=0).stop 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681081e0 msgr2=0x7f346819fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681081e0 0x7f346819fe50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f346000b9e0 tx=0x7f346000bda0 comp rx=0 tx=0).stop 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 shutdown_connections 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34681081e0 0x7f346819fe50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34540a2c20 0x7f34540a50e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3468102f80 0x7f3468198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 --2- 172.21.15.107:0/3936554728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34681058b0 0x7f3468198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 >> 172.21.15.107:0/3936554728 conn(0x7f34680fc990 msgr2=0x7f34681039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 shutdown_connections 2024-06-03T01:01:41.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:01:41.972+0000 7f346f3ac700 1 -- 172.21.15.107:0/3936554728 wait complete. 2024-06-03T01:01:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:42 smithi107 ceph-mon[36622]: from='osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T01:01:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:42 smithi107 ceph-mon[36622]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T01:01:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:42 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:42 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:42 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:42 smithi094 ceph-mon[32119]: from='osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T01:01:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:42 smithi094 ceph-mon[32119]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T01:01:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:42 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:42 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:42 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:42 smithi179 ceph-mon[37041]: from='osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T01:01:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:42 smithi179 ceph-mon[37041]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-03T01:01:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:42 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:42 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:42 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:01:42.619 DEBUG:teuthology.orchestra.run.smithi107:osd.7> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.7.service 2024-06-03T01:01:42.621 INFO:tasks.cephadm:Deploying osd.8 on smithi179 with /dev/vg_nvme/lv_2... 2024-06-03T01:01:42.621 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-03T01:01:42.869 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:01:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:43 smithi094 ceph-mon[32119]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:43 smithi094 ceph-mon[32119]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T01:01:43.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:43 smithi094 ceph-mon[32119]: from='osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:01:43.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:43 smithi094 ceph-mon[32119]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T01:01:43.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:43.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:43 smithi094 ceph-mon[32119]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:01:43.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:43 smithi094 ceph-mon[32119]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:01:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:43 smithi179 ceph-mon[37041]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:43 smithi179 ceph-mon[37041]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T01:01:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:43 smithi179 ceph-mon[37041]: from='osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:01:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:43 smithi179 ceph-mon[37041]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T01:01:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:43 smithi179 ceph-mon[37041]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:01:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:43 smithi179 ceph-mon[37041]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:01:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:43 smithi107 ceph-mon[36622]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:43 smithi107 ceph-mon[36622]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-03T01:01:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:43 smithi107 ceph-mon[36622]: from='osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:01:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:43 smithi107 ceph-mon[36622]: osdmap e50: 8 total, 7 up, 8 in 2024-06-03T01:01:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:43 smithi107 ceph-mon[36622]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:01:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:43 smithi107 ceph-mon[36622]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:01:44.128 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:01:44.010+0000 7f9cbd5af700 -1 osd.7 0 waiting for initial osdmap 2024-06-03T01:01:44.128 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:01:44.021+0000 7f9cb539a700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:01:44.379 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-mon[36622]: osdmap e51: 8 total, 7 up, 8 in 2024-06-03T01:01:44.380 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:44.380 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-mon[36622]: from='osd.7 ' entity='osd.7' 2024-06-03T01:01:44.380 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-mon[36622]: osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] boot 2024-06-03T01:01:44.380 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-mon[36622]: osdmap e52: 8 total, 8 up, 8 in 2024-06-03T01:01:44.380 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:44 smithi094 ceph-mon[32119]: osdmap e51: 8 total, 7 up, 8 in 2024-06-03T01:01:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:44 smithi094 ceph-mon[32119]: from='osd.7 ' entity='osd.7' 2024-06-03T01:01:44.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:44 smithi094 ceph-mon[32119]: osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] boot 2024-06-03T01:01:44.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:44 smithi094 ceph-mon[32119]: osdmap e52: 8 total, 8 up, 8 in 2024-06-03T01:01:44.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:44 smithi179 ceph-mon[37041]: osdmap e51: 8 total, 7 up, 8 in 2024-06-03T01:01:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:44 smithi179 ceph-mon[37041]: from='osd.7 ' entity='osd.7' 2024-06-03T01:01:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:44 smithi179 ceph-mon[37041]: osd.7 [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] boot 2024-06-03T01:01:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:44 smithi179 ceph-mon[37041]: osdmap e52: 8 total, 8 up, 8 in 2024-06-03T01:01:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:01:44.837 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-03T01:01:44.837 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-03T01:01:44.837 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-03T01:01:44.837 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-03T01:01:44.837 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112996 s, 928 MB/s 2024-06-03T01:01:44.837 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-03T01:01:45.486 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:45 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:01:45.486 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:45 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:01:45.486 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:45 smithi179 ceph-mon[37041]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:45.486 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:45 smithi179 ceph-mon[37041]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T01:01:45.510 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-06-03T01:01:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:45 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:01:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:45 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:01:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:45 smithi107 ceph-mon[36622]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:45 smithi107 ceph-mon[36622]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T01:01:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:45 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:01:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:45 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:01:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:45 smithi094 ceph-mon[32119]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-06-03T01:01:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:45 smithi094 ceph-mon[32119]: osdmap e53: 8 total, 8 up, 8 in 2024-06-03T01:01:45.752 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:01:46.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi107 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi107 to 5696M 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:46 smithi179 ceph-mon[37041]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T01:01:47.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi107 2024-06-03T01:01:47.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:47.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi107 to 5696M 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:47.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:46 smithi107 ceph-mon[36622]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T01:01:47.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi107 2024-06-03T01:01:47.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:47.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:47.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:01:47.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi107 to 5696M 2024-06-03T01:01:47.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:47.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:01:47.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:01:47.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:47.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:46 smithi094 ceph-mon[32119]: osdmap e54: 8 total, 8 up, 8 in 2024-06-03T01:01:47.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- 172.21.15.179:0/1534017880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 msgr2=0x7f911810a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:47.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 --2- 172.21.15.179:0/1534017880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911810a660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f911000b0a0 tx=0x7f911000b3b0 comp rx=0 tx=0).stop 2024-06-03T01:01:47.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- 172.21.15.179:0/1534017880 shutdown_connections 2024-06-03T01:01:47.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 --2- 172.21.15.179:0/1534017880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911810a660 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:47.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 --2- 172.21.15.179:0/1534017880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9118103010 0x7f9118105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:47.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 --2- 172.21.15.179:0/1534017880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118105940 0x7f9118107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- 172.21.15.179:0/1534017880 >> 172.21.15.179:0/1534017880 conn(0x7f91180fc9c0 msgr2=0x7f91180fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- 172.21.15.179:0/1534017880 shutdown_connections 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- 172.21.15.179:0/1534017880 wait complete. 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 Processor -- start 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- start start 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118103010 0x7f91181987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9118105940 0x7f9118198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f911810e1b0 con 0x7f9118105940 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f911810e030 con 0x7f9118103010 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.398+0000 7f9120443700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f911810e330 con 0x7f9118108270 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:01:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54360/0 (socket says 172.21.15.179:54360) 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 -- 172.21.15.179:0/4110903471 learned_addr learned my addr 172.21.15.179:0/4110903471 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e1df700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118103010 0x7f91181987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 -- 172.21.15.179:0/4110903471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118103010 msgr2=0x7f91181987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118103010 0x7f91181987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 -- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9118105940 msgr2=0x7f9118198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9118105940 0x7f9118198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 -- 172.21.15.179:0/4110903471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91181a0450 con 0x7f9118108270 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f911e9e0700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911819fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f911000d010 tx=0x7f911000be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f910b7fe700 1 -- 172.21.15.179:0/4110903471 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110021070 con 0x7f9118108270 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f910b7fe700 1 -- 172.21.15.179:0/4110903471 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9110007d00 con 0x7f9118108270 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91181a06e0 con 0x7f9118108270 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f910b7fe700 1 -- 172.21.15.179:0/4110903471 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911001d6b0 con 0x7f9118108270 2024-06-03T01:01:47.402 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.399+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91181a0bc0 con 0x7f9118108270 2024-06-03T01:01:47.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.400+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9118069180 con 0x7f9118108270 2024-06-03T01:01:47.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.401+0000 7f910b7fe700 1 -- 172.21.15.179:0/4110903471 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9110003a00 con 0x7f9118108270 2024-06-03T01:01:47.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.402+0000 7f910b7fe700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f91040a2c20 0x7f91040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:01:47.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.402+0000 7f910b7fe700 1 -- 172.21.15.179:0/4110903471 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f91100cf7c0 con 0x7f9118108270 2024-06-03T01:01:47.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.402+0000 7f911e1df700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f91040a2c20 0x7f91040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:01:47.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.403+0000 7f911e1df700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f91040a2c20 0x7f91040a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f910c009a10 tx=0x7f910c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:01:47.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.406+0000 7f910b7fe700 1 -- 172.21.15.179:0/4110903471 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f911009d2f0 con 0x7f9118108270 2024-06-03T01:01:47.609 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:01:47.608+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f91180628d0 con 0x7f91040a2c20 2024-06-03T01:01:48.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:47 smithi107 ceph-mon[36622]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-03T01:01:48.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:01:48.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:01:48.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:48.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:47 smithi094 ceph-mon[32119]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-03T01:01:48.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:01:48.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:01:48.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:47 smithi179 ceph-mon[37041]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-06-03T01:01:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:01:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:01:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:48 smithi179 ceph-mon[37041]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:01:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:48 smithi179 ceph-mon[37041]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T01:01:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:48 smithi179 ceph-mon[37041]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:48 smithi107 ceph-mon[36622]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:01:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:48 smithi107 ceph-mon[36622]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T01:01:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:48 smithi107 ceph-mon[36622]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:49.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:48 smithi094 ceph-mon[32119]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:01:49.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:48 smithi094 ceph-mon[32119]: osdmap e55: 8 total, 8 up, 8 in 2024-06-03T01:01:49.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:48 smithi094 ceph-mon[32119]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:49 smithi179 ceph-mon[37041]: osdmap e56: 8 total, 8 up, 8 in 2024-06-03T01:01:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:49 smithi107 ceph-mon[36622]: osdmap e56: 8 total, 8 up, 8 in 2024-06-03T01:01:50.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:49 smithi094 ceph-mon[32119]: osdmap e56: 8 total, 8 up, 8 in 2024-06-03T01:01:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:50 smithi107 ceph-mon[36622]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:50 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/1817715622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]: dispatch 2024-06-03T01:01:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:50 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]: dispatch 2024-06-03T01:01:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:50 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]': finished 2024-06-03T01:01:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:50 smithi107 ceph-mon[36622]: osdmap e57: 9 total, 8 up, 9 in 2024-06-03T01:01:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:01:51.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:50 smithi094 ceph-mon[32119]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:51.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:50 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/1817715622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]: dispatch 2024-06-03T01:01:51.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:50 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]: dispatch 2024-06-03T01:01:51.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:50 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]': finished 2024-06-03T01:01:51.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:50 smithi094 ceph-mon[32119]: osdmap e57: 9 total, 8 up, 9 in 2024-06-03T01:01:51.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:01:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:50 smithi179 ceph-mon[37041]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:50 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/1817715622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]: dispatch 2024-06-03T01:01:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:50 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]: dispatch 2024-06-03T01:01:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:50 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07303dac-9a89-4c1a-8555-07075f29ed66"}]': finished 2024-06-03T01:01:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:50 smithi179 ceph-mon[37041]: osdmap e57: 9 total, 8 up, 9 in 2024-06-03T01:01:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:01:52.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:51 smithi107 ceph-mon[36622]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-03T01:01:52.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:51 smithi107 ceph-mon[36622]: Cluster is now healthy 2024-06-03T01:01:52.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:51 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/2951997428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:01:52.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:51 smithi094 ceph-mon[32119]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-03T01:01:52.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:51 smithi094 ceph-mon[32119]: Cluster is now healthy 2024-06-03T01:01:52.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:51 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/2951997428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:01:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:51 smithi179 ceph-mon[37041]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-06-03T01:01:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:51 smithi179 ceph-mon[37041]: Cluster is now healthy 2024-06-03T01:01:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:51 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/2951997428' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:01:53.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:52 smithi107 ceph-mon[36622]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:53.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:52 smithi094 ceph-mon[32119]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:53.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:52 smithi179 ceph-mon[37041]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:55.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:55 smithi107 ceph-mon[36622]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:55.536 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:55 smithi179 ceph-mon[37041]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:55 smithi094 ceph-mon[32119]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:57.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:57 smithi179 ceph-mon[37041]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:57 smithi107 ceph-mon[36622]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:57 smithi094 ceph-mon[32119]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:58.525 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T01:01:58.525 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:58 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T01:01:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:58 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-03T01:01:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:58 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:01:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:59 smithi107 ceph-mon[36622]: Deploying daemon osd.8 on smithi179 2024-06-03T01:01:59.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:01:59 smithi107 ceph-mon[36622]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:59 smithi094 ceph-mon[32119]: Deploying daemon osd.8 on smithi179 2024-06-03T01:01:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:01:59 smithi094 ceph-mon[32119]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:01:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:59 smithi179 ceph-mon[37041]: Deploying daemon osd.8 on smithi179 2024-06-03T01:01:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:01:59 smithi179 ceph-mon[37041]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:01.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:01 smithi107 ceph-mon[36622]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:01.555 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:01 smithi179 ceph-mon[37041]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:01 smithi094 ceph-mon[32119]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:03.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:03 smithi094 ceph-mon[32119]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:03 smithi179 ceph-mon[37041]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:03.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:03 smithi107 ceph-mon[36622]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:04 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:04 smithi107 ceph-mon[36622]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:04 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:04 smithi094 ceph-mon[32119]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:04 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:04 smithi179 ceph-mon[37041]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:07.495 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 8 on host 'smithi179' 2024-06-03T01:02:07.496 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.495+0000 7f910b7fe700 1 -- 172.21.15.179:0/4110903471 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f91180628d0 con 0x7f91040a2c20 2024-06-03T01:02:07.497 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f91040a2c20 msgr2=0x7f91040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f91040a2c20 0x7f91040a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f910c009a10 tx=0x7f910c009190 comp rx=0 tx=0).stop 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 msgr2=0x7f911819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911819fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f911000d010 tx=0x7f911000be60 comp rx=0 tx=0).stop 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 shutdown_connections 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9118103010 0x7f91181987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f91040a2c20 0x7f91040a50e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9118108270 0x7f911819fdb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 --2- 172.21.15.179:0/4110903471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9118105940 0x7f9118198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 >> 172.21.15.179:0/4110903471 conn(0x7f91180fc9c0 msgr2=0x7f9118103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 shutdown_connections 2024-06-03T01:02:07.498 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:02:07.498+0000 7f9120443700 1 -- 172.21.15.179:0/4110903471 wait complete. 2024-06-03T01:02:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:07 smithi107 ceph-mon[36622]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:07.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:07 smithi094 ceph-mon[32119]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:07.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:07 smithi179 ceph-mon[37041]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:08.194 DEBUG:teuthology.orchestra.run.smithi179:osd.8> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.8.service 2024-06-03T01:02:08.197 INFO:tasks.cephadm:Deploying osd.9 on smithi094 with /dev/vg_nvme/lv_1... 2024-06-03T01:02:08.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T01:02:08.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:08 smithi179 ceph-mon[37041]: from='osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T01:02:08.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:08 smithi179 ceph-mon[37041]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T01:02:08.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:08 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:08.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:08 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:08.446 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:08 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:08 smithi107 ceph-mon[36622]: from='osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T01:02:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:08 smithi107 ceph-mon[36622]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T01:02:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:08 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:08 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:08 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:08 smithi094 ceph-mon[32119]: from='osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T01:02:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:08 smithi094 ceph-mon[32119]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-03T01:02:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:08.632 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:02:09.462 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-mon[37041]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:09.462 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-mon[37041]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T01:02:09.462 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-mon[37041]: from='osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:02:09.462 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-mon[37041]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T01:02:09.462 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:09.462 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-mon[37041]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:02:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:09 smithi107 ceph-mon[36622]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:09 smithi107 ceph-mon[36622]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T01:02:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:09 smithi107 ceph-mon[36622]: from='osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:02:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:09 smithi107 ceph-mon[36622]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T01:02:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:09 smithi107 ceph-mon[36622]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:02:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:09 smithi094 ceph-mon[32119]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:09 smithi094 ceph-mon[32119]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-03T01:02:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:09 smithi094 ceph-mon[32119]: from='osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:02:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:09 smithi094 ceph-mon[32119]: osdmap e58: 9 total, 8 up, 9 in 2024-06-03T01:02:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:09 smithi094 ceph-mon[32119]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:02:10.106 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:02:09.968+0000 7fca82a8c700 -1 osd.8 0 waiting for initial osdmap 2024-06-03T01:02:10.106 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:02:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:02:09.982+0000 7fca7a877700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:02:10.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:10 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:02:10.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:10 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:02:10.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:10 smithi094 ceph-mon[32119]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:02:10.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:10 smithi094 ceph-mon[32119]: osdmap e59: 9 total, 8 up, 9 in 2024-06-03T01:02:10.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:10.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:10.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:10 smithi094 ceph-mon[32119]: from='osd.8 ' entity='osd.8' 2024-06-03T01:02:10.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:10 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:02:10.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:10 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:02:10.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:10 smithi107 ceph-mon[36622]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:02:10.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:10 smithi107 ceph-mon[36622]: osdmap e59: 9 total, 8 up, 9 in 2024-06-03T01:02:10.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:10.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:10.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:10 smithi107 ceph-mon[36622]: from='osd.8 ' entity='osd.8' 2024-06-03T01:02:10.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:10 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:02:10.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:10 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:02:10.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:10 smithi179 ceph-mon[37041]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:02:10.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:10 smithi179 ceph-mon[37041]: osdmap e59: 9 total, 8 up, 9 in 2024-06-03T01:02:10.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:10.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:10.575 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:10 smithi179 ceph-mon[37041]: from='osd.8 ' entity='osd.8' 2024-06-03T01:02:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:11 smithi094 ceph-mon[32119]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:11 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:11 smithi094 ceph-mon[32119]: osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] boot 2024-06-03T01:02:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:11 smithi094 ceph-mon[32119]: osdmap e60: 9 total, 9 up, 9 in 2024-06-03T01:02:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:11 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:11 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:11 smithi179 ceph-mon[37041]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:11 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:11 smithi179 ceph-mon[37041]: osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] boot 2024-06-03T01:02:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:11 smithi179 ceph-mon[37041]: osdmap e60: 9 total, 9 up, 9 in 2024-06-03T01:02:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:11 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:11 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:11 smithi107 ceph-mon[36622]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-06-03T01:02:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:11 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:11 smithi107 ceph-mon[36622]: osd.8 [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] boot 2024-06-03T01:02:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:11 smithi107 ceph-mon[36622]: osdmap e60: 9 total, 9 up, 9 in 2024-06-03T01:02:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:11 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:02:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:11 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:11.808 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T01:02:11.808 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T01:02:11.808 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-06-03T01:02:11.809 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-06-03T01:02:11.809 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0484762 s, 216 MB/s 2024-06-03T01:02:11.809 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-06-03T01:02:12.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi179 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi179 to 5696M 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:12.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi179 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi179 to 5696M 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:12.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:12.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-06-03T01:02:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi179 2024-06-03T01:02:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: osdmap e61: 9 total, 9 up, 9 in 2024-06-03T01:02:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:12.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi179 to 5696M 2024-06-03T01:02:12.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:12.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:12.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:12.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:13.103 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:02:13.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:13 smithi094 ceph-mon[32119]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:13.593 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:13 smithi094 ceph-mon[32119]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T01:02:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:13 smithi179 ceph-mon[37041]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:13 smithi179 ceph-mon[37041]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T01:02:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:13 smithi107 ceph-mon[36622]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:13 smithi107 ceph-mon[36622]: osdmap e62: 9 total, 9 up, 9 in 2024-06-03T01:02:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:14 smithi094 ceph-mon[32119]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T01:02:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:14 smithi094 ceph-mon[32119]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:14.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:14 smithi179 ceph-mon[37041]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T01:02:14.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:14 smithi179 ceph-mon[37041]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:14 smithi107 ceph-mon[36622]: osdmap e63: 9 total, 9 up, 9 in 2024-06-03T01:02:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:14 smithi107 ceph-mon[36622]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:15.501 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:15 smithi094 ceph-mon[32119]: osdmap e64: 9 total, 9 up, 9 in 2024-06-03T01:02:15.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 -- 172.21.15.94:0/1658161479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 msgr2=0x7f264810a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 --2- 172.21.15.94:0/1658161479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 0x7f264810a860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2640009f90 tx=0x7f264000b330 comp rx=0 tx=0).stop 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 -- 172.21.15.94:0/1658161479 shutdown_connections 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 --2- 172.21.15.94:0/1658161479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 0x7f2648102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 --2- 172.21.15.94:0/1658161479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 0x7f264810a860 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 --2- 172.21.15.94:0/1658161479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 0x7f2648103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 -- 172.21.15.94:0/1658161479 >> 172.21.15.94:0/1658161479 conn(0x7f26480fd9d0 msgr2=0x7f26480ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 -- 172.21.15.94:0/1658161479 shutdown_connections 2024-06-03T01:02:15.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.518+0000 7f264ca00700 1 -- 172.21.15.94:0/1658161479 wait complete. 2024-06-03T01:02:15.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.519+0000 7f264ca00700 1 Processor -- start 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.519+0000 7f264ca00700 1 -- start start 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f264ca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 0x7f264819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f264ca00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 0x7f264819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f264ca00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 0x7f26481a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f264ca00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f264810e030 con 0x7f2648103610 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f264ca00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f264810deb0 con 0x7f2648102410 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f264ca00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f264810e1b0 con 0x7f2648103fb0 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f2646a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 0x7f264819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f2647400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 0x7f264819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f2646a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 0x7f264819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45620/0 (socket says 172.21.15.94:45620) 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f2647400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 0x7f264819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42540/0 (socket says 172.21.15.94:42540) 2024-06-03T01:02:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f2646a00700 1 -- 172.21.15.94:0/4045234104 learned_addr learned my addr 172.21.15.94:0/4045234104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:02:15.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.520+0000 7f2647e00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 0x7f26481a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:02:15.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f2646a00700 1 -- 172.21.15.94:0/4045234104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 msgr2=0x7f26481a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:15.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f2646a00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 0x7f26481a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:15.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f2646a00700 1 -- 172.21.15.94:0/4045234104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 msgr2=0x7f264819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:15.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f2646a00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 0x7f264819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:15.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f2646a00700 1 -- 172.21.15.94:0/4045234104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26481a23d0 con 0x7f2648103610 2024-06-03T01:02:15.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f2647400700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 0x7f264819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:02:15.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f2646a00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 0x7f264819acb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f263400e9f0 tx=0x7f263400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:02:15.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f263fe00700 1 -- 172.21.15.94:0/4045234104 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634004a30 con 0x7f2648103610 2024-06-03T01:02:15.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26481a26c0 con 0x7f2648103610 2024-06-03T01:02:15.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f263fe00700 1 -- 172.21.15.94:0/4045234104 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2634005500 con 0x7f2648103610 2024-06-03T01:02:15.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f263fe00700 1 -- 172.21.15.94:0/4045234104 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263400cd60 con 0x7f2648103610 2024-06-03T01:02:15.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.521+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26481a2c50 con 0x7f2648103610 2024-06-03T01:02:15.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.522+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2648069180 con 0x7f2648103610 2024-06-03T01:02:15.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.525+0000 7f263fe00700 1 -- 172.21.15.94:0/4045234104 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2634004c60 con 0x7f2648103610 2024-06-03T01:02:15.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.526+0000 7f263fe00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f262c0a2ca0 0x7f262c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:15.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.527+0000 7f263fe00700 1 -- 172.21.15.94:0/4045234104 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f26340cea00 con 0x7f2648103610 2024-06-03T01:02:15.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.527+0000 7f2647400700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f262c0a2ca0 0x7f262c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:02:15.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.528+0000 7f2647400700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f262c0a2ca0 0x7f262c0a5160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2648103470 tx=0x7f2638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:02:15.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.529+0000 7f263fe00700 1 -- 172.21.15.94:0/4045234104 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2634053850 con 0x7f2648103610 2024-06-03T01:02:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:15 smithi179 ceph-mon[37041]: osdmap e64: 9 total, 9 up, 9 in 2024-06-03T01:02:15.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:15.765+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f26481a2f10 con 0x7f262c0a2ca0 2024-06-03T01:02:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:15 smithi107 ceph-mon[36622]: osdmap e64: 9 total, 9 up, 9 in 2024-06-03T01:02:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:16 smithi094 ceph-mon[32119]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:02:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:02:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:02:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:16 smithi094 ceph-mon[32119]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-03T01:02:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:16 smithi179 ceph-mon[37041]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:02:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:02:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:02:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:16 smithi179 ceph-mon[37041]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-03T01:02:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:16 smithi107 ceph-mon[36622]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:02:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:02:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:02:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:16 smithi107 ceph-mon[36622]: pgmap v225: 1 pgs: 1 peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-03T01:02:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:19 smithi107 ceph-mon[36622]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:02:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:19 smithi094 ceph-mon[32119]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:02:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:19 smithi179 ceph-mon[37041]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:02:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:20 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2854336883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]: dispatch 2024-06-03T01:02:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:20 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]: dispatch 2024-06-03T01:02:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:20 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]': finished 2024-06-03T01:02:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:20 smithi107 ceph-mon[36622]: osdmap e65: 10 total, 9 up, 10 in 2024-06-03T01:02:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:20 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:20.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:20 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2854336883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]: dispatch 2024-06-03T01:02:20.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:20 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]: dispatch 2024-06-03T01:02:20.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:20 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]': finished 2024-06-03T01:02:20.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:20 smithi094 ceph-mon[32119]: osdmap e65: 10 total, 9 up, 10 in 2024-06-03T01:02:20.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:20 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:20 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2854336883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]: dispatch 2024-06-03T01:02:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:20 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]: dispatch 2024-06-03T01:02:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:20 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "126f498f-8208-42d7-bc3e-9dcc5f200906"}]': finished 2024-06-03T01:02:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:20 smithi179 ceph-mon[37041]: osdmap e65: 10 total, 9 up, 10 in 2024-06-03T01:02:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:20 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:21.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:21 smithi107 ceph-mon[36622]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-03T01:02:21.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:21 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2976271029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:02:21.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:21 smithi094 ceph-mon[32119]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-03T01:02:21.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:21 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2976271029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:02:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:21 smithi179 ceph-mon[37041]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-03T01:02:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:21 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2976271029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:02:23.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:23 smithi107 ceph-mon[36622]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T01:02:23.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:23 smithi094 ceph-mon[32119]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T01:02:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:23 smithi179 ceph-mon[37041]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-03T01:02:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:25 smithi094 ceph-mon[32119]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:25.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:25 smithi107 ceph-mon[36622]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:25 smithi179 ceph-mon[37041]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:27 smithi094 ceph-mon[32119]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:27 smithi179 ceph-mon[37041]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:27 smithi107 ceph-mon[36622]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:29 smithi094 ceph-mon[32119]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:29 smithi179 ceph-mon[37041]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:29 smithi107 ceph-mon[36622]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:30 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T01:02:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:30 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:30 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T01:02:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:30 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:30.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:30 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-03T01:02:30.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:30 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:31 smithi094 ceph-mon[32119]: Deploying daemon osd.9 on smithi094 2024-06-03T01:02:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:31 smithi094 ceph-mon[32119]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:31 smithi179 ceph-mon[37041]: Deploying daemon osd.9 on smithi094 2024-06-03T01:02:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:31 smithi179 ceph-mon[37041]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:31 smithi107 ceph-mon[36622]: Deploying daemon osd.9 on smithi094 2024-06-03T01:02:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:31 smithi107 ceph-mon[36622]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:33 smithi094 ceph-mon[32119]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:33 smithi179 ceph-mon[37041]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:33.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:33 smithi107 ceph-mon[36622]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:35 smithi094 ceph-mon[32119]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:35 smithi179 ceph-mon[37041]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:35.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:35 smithi107 ceph-mon[36622]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:36.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:36 smithi094 ceph-mon[32119]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:36 smithi107 ceph-mon[36622]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:36 smithi179 ceph-mon[37041]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:39 smithi107 ceph-mon[36622]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:39.327 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:39.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:39.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:39.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:39 smithi094 ceph-mon[32119]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:39 smithi179 ceph-mon[37041]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:41.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:41 smithi107 ceph-mon[36622]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:41.574 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:41 smithi094 ceph-mon[32119]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:41.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:41 smithi179 ceph-mon[37041]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:42 smithi107 ceph-mon[36622]: from='osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T01:02:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:42 smithi107 ceph-mon[36622]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T01:02:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:42 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:42 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:42 smithi094 ceph-mon[32119]: from='osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T01:02:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:42 smithi094 ceph-mon[32119]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T01:02:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:42 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:42 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:42 smithi179 ceph-mon[37041]: from='osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T01:02:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:42 smithi179 ceph-mon[37041]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-03T01:02:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:42 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:42 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:43.068 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 9 on host 'smithi094' 2024-06-03T01:02:43.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.067+0000 7f263fe00700 1 -- 172.21.15.94:0/4045234104 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26481a2f10 con 0x7f262c0a2ca0 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f262c0a2ca0 msgr2=0x7f262c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f262c0a2ca0 0x7f262c0a5160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2648103470 tx=0x7f2638009260 comp rx=0 tx=0).stop 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 msgr2=0x7f264819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 0x7f264819acb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f263400e9f0 tx=0x7f263400ed00 comp rx=0 tx=0).stop 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 shutdown_connections 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648102410 0x7f264819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f262c0a2ca0 0x7f262c0a5160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2648103fb0 0x7f26481a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 --2- 172.21.15.94:0/4045234104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2648103610 0x7f264819acb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.072+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 >> 172.21.15.94:0/4045234104 conn(0x7f26480fd9d0 msgr2=0x7f26480746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:02:43.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.073+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 shutdown_connections 2024-06-03T01:02:43.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:02:43.073+0000 7f264ca00700 1 -- 172.21.15.94:0/4045234104 wait complete. 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: from='osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T01:02:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: from='osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:02:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T01:02:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:02:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:43.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-03T01:02:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: from='osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:02:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: osdmap e66: 10 total, 9 up, 10 in 2024-06-03T01:02:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:43.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-06-03T01:02:43.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:43.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:43.867 DEBUG:teuthology.orchestra.run.smithi094:osd.9> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.9.service 2024-06-03T01:02:43.872 INFO:tasks.cephadm:Deploying osd.10 on smithi107 with /dev/vg_nvme/lv_1... 2024-06-03T01:02:43.872 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T01:02:44.077 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:02:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:02:44.048+0000 7fb9ea800700 -1 osd.9 0 waiting for initial osdmap 2024-06-03T01:02:44.077 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:02:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:02:44.076+0000 7fb9e0600700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:02:44.117 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:02:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:44 smithi094 ceph-mon[32119]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T01:02:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:44 smithi094 ceph-mon[32119]: osdmap e67: 10 total, 9 up, 10 in 2024-06-03T01:02:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:44 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:44.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:44 smithi094 ceph-mon[32119]: from='osd.9 ' entity='osd.9' 2024-06-03T01:02:44.497 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:44 smithi107 ceph-mon[36622]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T01:02:44.498 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:44 smithi107 ceph-mon[36622]: osdmap e67: 10 total, 9 up, 10 in 2024-06-03T01:02:44.498 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:44.498 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:44 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:44.498 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:44 smithi107 ceph-mon[36622]: from='osd.9 ' entity='osd.9' 2024-06-03T01:02:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:44 smithi179 ceph-mon[37041]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-06-03T01:02:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:44 smithi179 ceph-mon[37041]: osdmap e67: 10 total, 9 up, 10 in 2024-06-03T01:02:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:44 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:44 smithi179 ceph-mon[37041]: from='osd.9 ' entity='osd.9' 2024-06-03T01:02:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:45 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:02:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:45 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:02:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:45 smithi107 ceph-mon[36622]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:45 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:45 smithi107 ceph-mon[36622]: osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] boot 2024-06-03T01:02:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:45 smithi107 ceph-mon[36622]: osdmap e68: 10 total, 10 up, 10 in 2024-06-03T01:02:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:45 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:45 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:02:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:45 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:02:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:45 smithi094 ceph-mon[32119]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:45 smithi094 ceph-mon[32119]: osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] boot 2024-06-03T01:02:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:45 smithi094 ceph-mon[32119]: osdmap e68: 10 total, 10 up, 10 in 2024-06-03T01:02:45.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:45 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:02:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:45 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:02:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:45 smithi179 ceph-mon[37041]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-06-03T01:02:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:45 smithi179 ceph-mon[37041]: osd.9 [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] boot 2024-06-03T01:02:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:45 smithi179 ceph-mon[37041]: osdmap e68: 10 total, 10 up, 10 in 2024-06-03T01:02:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:02:46.237 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T01:02:46.237 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T01:02:46.237 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-06-03T01:02:46.237 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-06-03T01:02:46.237 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113232 s, 926 MB/s 2024-06-03T01:02:46.237 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-06-03T01:02:46.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-06-03T01:02:47.114 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:02:47.496 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:47 smithi107 ceph-mon[36622]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:47.497 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:47 smithi107 ceph-mon[36622]: osdmap e69: 10 total, 10 up, 10 in 2024-06-03T01:02:47.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:47 smithi094 ceph-mon[32119]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:47.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:47 smithi094 ceph-mon[32119]: osdmap e69: 10 total, 10 up, 10 in 2024-06-03T01:02:47.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:47 smithi179 ceph-mon[37041]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:47.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:47 smithi179 ceph-mon[37041]: osdmap e69: 10 total, 10 up, 10 in 2024-06-03T01:02:48.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 -- 172.21.15.107:0/1055649089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 msgr2=0x7f2660101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 --2- 172.21.15.107:0/1055649089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f2660101b30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2650009990 tx=0x7f2650009ca0 comp rx=0 tx=0).stop 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 -- 172.21.15.107:0/1055649089 shutdown_connections 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 --2- 172.21.15.107:0/1055649089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102070 0x7f2660109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 --2- 172.21.15.107:0/1055649089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2660102990 0x7f2660102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 --2- 172.21.15.107:0/1055649089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f2660101b30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 -- 172.21.15.107:0/1055649089 >> 172.21.15.107:0/1055649089 conn(0x7f26600fc100 msgr2=0x7f26600fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 -- 172.21.15.107:0/1055649089 shutdown_connections 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.781+0000 7f26651af700 1 -- 172.21.15.107:0/1055649089 wait complete. 2024-06-03T01:02:48.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.782+0000 7f26651af700 1 Processor -- start 2024-06-03T01:02:48.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.782+0000 7f26651af700 1 -- start start 2024-06-03T01:02:48.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f26651af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f26601987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:48.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f26651af700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2660102070 0x7f2660198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:48.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f26651af700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102990 0x7f266019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:48.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f26651af700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f266010d0b0 con 0x7f2660102990 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f26651af700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f266010cf30 con 0x7f26601016d0 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f26651af700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f266010d230 con 0x7f2660102070 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f26601987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f26601987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48582/0 (socket says 172.21.15.107:48582) 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 -- 172.21.15.107:0/3516712570 learned_addr learned my addr 172.21.15.107:0/3516712570 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265e59c700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2660102070 0x7f2660198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 -- 172.21.15.107:0/3516712570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2660102070 msgr2=0x7f2660198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2660102070 0x7f2660198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 -- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102990 msgr2=0x7f266019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102990 0x7f266019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 -- 172.21.15.107:0/3516712570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26601a0410 con 0x7f26601016d0 2024-06-03T01:02:48.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f265ed9d700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f26601987b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f264800d980 tx=0x7f264800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:02:48.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.783+0000 7f2657fff700 1 -- 172.21.15.107:0/3516712570 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648017070 con 0x7f26601016d0 2024-06-03T01:02:48.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.784+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26601a0700 con 0x7f26601016d0 2024-06-03T01:02:48.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.784+0000 7f2657fff700 1 -- 172.21.15.107:0/3516712570 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648009e10 con 0x7f26601016d0 2024-06-03T01:02:48.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.784+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26601a0c70 con 0x7f26601016d0 2024-06-03T01:02:48.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.784+0000 7f2657fff700 1 -- 172.21.15.107:0/3516712570 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648020060 con 0x7f26601016d0 2024-06-03T01:02:48.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.785+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2660069180 con 0x7f26601016d0 2024-06-03T01:02:48.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.786+0000 7f2657fff700 1 -- 172.21.15.107:0/3516712570 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2648010710 con 0x7f26601016d0 2024-06-03T01:02:48.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.786+0000 7f2657fff700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f264c0a2c20 0x7f264c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:02:48.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.786+0000 7f2657fff700 1 -- 172.21.15.107:0/3516712570 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f26480ce6e0 con 0x7f26601016d0 2024-06-03T01:02:48.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.787+0000 7f265e59c700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f264c0a2c20 0x7f264c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:02:48.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.788+0000 7f265e59c700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f264c0a2c20 0x7f264c0a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f265000b5c0 tx=0x7f2650005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:02:48.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.791+0000 7f2657fff700 1 -- 172.21.15.107:0/3516712570 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f264809be40 con 0x7f26601016d0 2024-06-03T01:02:48.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:02:48.993+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f26600fe270 con 0x7f264c0a2c20 2024-06-03T01:02:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:02:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:02:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:49.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:49 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:02:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:02:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:49 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.638 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:49.638 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:02:49.638 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:02:49.638 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:49.638 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.638 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:49.639 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:49 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:50 smithi094 ceph-mon[32119]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:02:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:50 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi094 2024-06-03T01:02:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:50 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi094 to 4272M 2024-06-03T01:02:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:50.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:50.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:50.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:50 smithi179 ceph-mon[37041]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:02:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:50 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi094 2024-06-03T01:02:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:50 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi094 to 4272M 2024-06-03T01:02:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:50.659 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:50 smithi107 ceph-mon[36622]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:02:50.659 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:50 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi094 2024-06-03T01:02:50.659 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:50 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi094 to 4272M 2024-06-03T01:02:50.659 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:02:50.660 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:02:50.660 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:02:50.660 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:02:51.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:51 smithi094 ceph-mon[32119]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:51 smithi179 ceph-mon[37041]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:51 smithi107 ceph-mon[36622]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:52 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/299989850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]: dispatch 2024-06-03T01:02:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:52 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]: dispatch 2024-06-03T01:02:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:52 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]': finished 2024-06-03T01:02:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:52 smithi107 ceph-mon[36622]: osdmap e70: 11 total, 10 up, 11 in 2024-06-03T01:02:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:52 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:02:52.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:52 smithi107 ceph-mon[36622]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:52 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/299989850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]: dispatch 2024-06-03T01:02:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:52 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]: dispatch 2024-06-03T01:02:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:52 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]': finished 2024-06-03T01:02:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:52 smithi094 ceph-mon[32119]: osdmap e70: 11 total, 10 up, 11 in 2024-06-03T01:02:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:52 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:02:52.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:52 smithi094 ceph-mon[32119]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:52 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/299989850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]: dispatch 2024-06-03T01:02:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:52 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]: dispatch 2024-06-03T01:02:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:52 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "070f388a-c6f7-4438-86bc-57d8f08e4175"}]': finished 2024-06-03T01:02:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:52 smithi179 ceph-mon[37041]: osdmap e70: 11 total, 10 up, 11 in 2024-06-03T01:02:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:52 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:02:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:52 smithi179 ceph-mon[37041]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:53.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:53 smithi094 ceph-mon[32119]: from='client.? 172.21.15.107:0/518143807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:02:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:53 smithi179 ceph-mon[37041]: from='client.? 172.21.15.107:0/518143807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:02:53.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:53 smithi107 ceph-mon[36622]: from='client.? 172.21.15.107:0/518143807' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:02:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:54 smithi094 ceph-mon[32119]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:54 smithi179 ceph-mon[37041]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:54 smithi107 ceph-mon[36622]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:57.532 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:57 smithi107 ceph-mon[36622]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:57 smithi094 ceph-mon[32119]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:57 smithi179 ceph-mon[37041]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:02:59 smithi107 ceph-mon[36622]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:02:59 smithi094 ceph-mon[32119]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:02:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:02:59 smithi179 ceph-mon[37041]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T01:03:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T01:03:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-03T01:03:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:01.592 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:01 smithi107 ceph-mon[36622]: Deploying daemon osd.10 on smithi107 2024-06-03T01:03:01.592 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:01 smithi107 ceph-mon[36622]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:01 smithi094 ceph-mon[32119]: Deploying daemon osd.10 on smithi107 2024-06-03T01:03:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:01 smithi094 ceph-mon[32119]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:01 smithi179 ceph-mon[37041]: Deploying daemon osd.10 on smithi107 2024-06-03T01:03:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:01 smithi179 ceph-mon[37041]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:03.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:03 smithi107 ceph-mon[36622]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:03.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:03 smithi094 ceph-mon[32119]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:03 smithi179 ceph-mon[37041]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:05 smithi094 ceph-mon[32119]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:05 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:05 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:05 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:05 smithi179 ceph-mon[37041]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:05 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:05 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:05 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:05 smithi107 ceph-mon[36622]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:05 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:05 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:05 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:07.444 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:07 smithi107 ceph-mon[36622]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:07 smithi094 ceph-mon[32119]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:07 smithi179 ceph-mon[37041]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:09.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.078+0000 7f2657fff700 1 -- 172.21.15.107:0/3516712570 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f26600fe270 con 0x7f264c0a2c20 2024-06-03T01:03:09.079 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 10 on host 'smithi107' 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f264c0a2c20 msgr2=0x7f264c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f264c0a2c20 0x7f264c0a50e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f265000b5c0 tx=0x7f2650005fb0 comp rx=0 tx=0).stop 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 msgr2=0x7f26601987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f26601987b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f264800d980 tx=0x7f264800dc90 comp rx=0 tx=0).stop 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 shutdown_connections 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26601016d0 0x7f26601987b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f264c0a2c20 0x7f264c0a50e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2660102990 0x7f266019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 --2- 172.21.15.107:0/3516712570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2660102070 0x7f2660198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:09.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.080+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 >> 172.21.15.107:0/3516712570 conn(0x7f26600fc100 msgr2=0x7f26600fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:09.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.081+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 shutdown_connections 2024-06-03T01:03:09.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-06-03T01:03:09.081+0000 7f26651af700 1 -- 172.21.15.107:0/3516712570 wait complete. 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T01:03:09.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T01:03:09.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:09 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:09 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T01:03:09.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-03T01:03:09.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:09.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:09 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:09.715 DEBUG:teuthology.orchestra.run.smithi107:osd.10> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.10.service 2024-06-03T01:03:09.717 INFO:tasks.cephadm:Deploying osd.11 on smithi179 with /dev/vg_nvme/lv_1... 2024-06-03T01:03:09.717 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-03T01:03:09.963 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:03:11.144 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-mon[36622]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T01:03:11.145 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-mon[36622]: from='osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:03:11.145 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-mon[36622]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T01:03:11.145 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:11.145 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-mon[36622]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:03:11.145 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-mon[36622]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:11.330 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:11 smithi179 ceph-mon[37041]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T01:03:11.331 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:11 smithi179 ceph-mon[37041]: from='osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:03:11.331 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:11 smithi179 ceph-mon[37041]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T01:03:11.331 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:11 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:11.331 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:11 smithi179 ceph-mon[37041]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:03:11.331 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:11 smithi179 ceph-mon[37041]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:11.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:11 smithi094 ceph-mon[32119]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-03T01:03:11.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:11 smithi094 ceph-mon[32119]: from='osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:03:11.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:11 smithi094 ceph-mon[32119]: osdmap e71: 11 total, 10 up, 11 in 2024-06-03T01:03:11.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:11 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:11.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:11 smithi094 ceph-mon[32119]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-06-03T01:03:11.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:11 smithi094 ceph-mon[32119]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-06-03T01:03:12.077 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-03T01:03:12.077 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-03T01:03:12.078 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-06-03T01:03:12.078 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-06-03T01:03:12.078 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014774 s, 710 MB/s 2024-06-03T01:03:12.078 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-06-03T01:03:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:12 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:03:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:12 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:03:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:12 smithi107 ceph-mon[36622]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:03:12.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:12 smithi107 ceph-mon[36622]: osdmap e72: 11 total, 10 up, 11 in 2024-06-03T01:03:12.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:12.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:12 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:12.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:12 smithi107 ceph-mon[36622]: from='osd.10 ' entity='osd.10' 2024-06-03T01:03:12.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:03:11.804+0000 7ff05416c700 -1 osd.10 0 waiting for initial osdmap 2024-06-03T01:03:12.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:03:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:03:11.819+0000 7ff04bf57700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:03:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:12 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:03:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:12 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:03:12.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:12 smithi094 ceph-mon[32119]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:03:12.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:12 smithi094 ceph-mon[32119]: osdmap e72: 11 total, 10 up, 11 in 2024-06-03T01:03:12.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:12.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:12 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:12.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:12 smithi094 ceph-mon[32119]: from='osd.10 ' entity='osd.10' 2024-06-03T01:03:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:12 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:03:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:12 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:03:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:12 smithi179 ceph-mon[37041]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-06-03T01:03:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:12 smithi179 ceph-mon[37041]: osdmap e72: 11 total, 10 up, 11 in 2024-06-03T01:03:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:12 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:12 smithi179 ceph-mon[37041]: from='osd.10 ' entity='osd.10' 2024-06-03T01:03:12.771 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-06-03T01:03:13.013 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:03:13.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] boot 2024-06-03T01:03:13.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: osdmap e73: 11 total, 11 up, 11 in 2024-06-03T01:03:13.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:13.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:13.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:13.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:13.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T01:03:13.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] boot 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: osdmap e73: 11 total, 11 up, 11 in 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:13.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:13.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T01:03:13.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: osd.10 [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] boot 2024-06-03T01:03:13.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: osdmap e73: 11 total, 11 up, 11 in 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:13.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:13.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: osdmap e74: 11 total, 11 up, 11 in 2024-06-03T01:03:13.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:14.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:14 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi107 2024-06-03T01:03:14.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:14 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi107 to 4272M 2024-06-03T01:03:14.184 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:14 smithi179 ceph-mon[37041]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T01:03:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:14 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi107 2024-06-03T01:03:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:14 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi107 to 4272M 2024-06-03T01:03:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:14 smithi094 ceph-mon[32119]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T01:03:14.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:14 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi107 2024-06-03T01:03:14.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:14 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi107 to 4272M 2024-06-03T01:03:14.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:14 smithi107 ceph-mon[36622]: osdmap e75: 11 total, 11 up, 11 in 2024-06-03T01:03:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 -- 172.21.15.179:0/2414986828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1441040c0 msgr2=0x7fe14410a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 --2- 172.21.15.179:0/2414986828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1441040c0 0x7fe14410a970 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b0a0 tx=0x7fe13c00b3b0 comp rx=0 tx=0).stop 2024-06-03T01:03:14.675 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 -- 172.21.15.179:0/2414986828 shutdown_connections 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 --2- 172.21.15.179:0/2414986828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe144102520 0x7fe144102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 --2- 172.21.15.179:0/2414986828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1441040c0 0x7fe14410a970 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 --2- 172.21.15.179:0/2414986828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe144103720 0x7fe144103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 -- 172.21.15.179:0/2414986828 >> 172.21.15.179:0/2414986828 conn(0x7fe1440fdae0 msgr2=0x7fe1440fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 -- 172.21.15.179:0/2414986828 shutdown_connections 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 -- 172.21.15.179:0/2414986828 wait complete. 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.673+0000 7fe14a335700 1 Processor -- start 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe14a335700 1 -- start start 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe14a335700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe144102520 0x7fe144198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe14a335700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe144103720 0x7fe144198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe14a335700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1441040c0 0x7fe14419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe14a335700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe14410e250 con 0x7fe144102520 2024-06-03T01:03:14.676 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe14a335700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe14410e0d0 con 0x7fe1441040c0 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe14a335700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe14410e3d0 con 0x7fe144103720 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe144103720 0x7fe144198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe144103720 0x7fe144198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:34606/0 (socket says 172.21.15.179:34606) 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 -- 172.21.15.179:0/3924901306 learned_addr learned my addr 172.21.15.179:0/3924901306 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1488d2700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1441040c0 0x7fe14419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 -- 172.21.15.179:0/3924901306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1441040c0 msgr2=0x7fe14419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1441040c0 0x7fe14419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 -- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe144102520 msgr2=0x7fe144198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe144102520 0x7fe144198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 -- 172.21.15.179:0/3924901306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1441a0490 con 0x7fe144103720 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.674+0000 7fe1437fe700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe144103720 0x7fe144198d70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b560 tx=0x7fe13800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.675+0000 7fe1417fa700 1 -- 172.21.15.179:0/3924901306 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138015020 con 0x7fe144103720 2024-06-03T01:03:14.677 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.675+0000 7fe1417fa700 1 -- 172.21.15.179:0/3924901306 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe138011d10 con 0x7fe144103720 2024-06-03T01:03:14.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.675+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1441a0780 con 0x7fe144103720 2024-06-03T01:03:14.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.675+0000 7fe1417fa700 1 -- 172.21.15.179:0/3924901306 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13800f2e0 con 0x7fe144103720 2024-06-03T01:03:14.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.675+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1441a0d40 con 0x7fe144103720 2024-06-03T01:03:14.678 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.676+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe144069180 con 0x7fe144103720 2024-06-03T01:03:14.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.677+0000 7fe1417fa700 1 -- 172.21.15.179:0/3924901306 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe138018020 con 0x7fe144103720 2024-06-03T01:03:14.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.677+0000 7fe1417fa700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe12c0a2c20 0x7fe12c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:14.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.677+0000 7fe1417fa700 1 -- 172.21.15.179:0/3924901306 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7fe1380ce220 con 0x7fe144103720 2024-06-03T01:03:14.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.678+0000 7fe143fff700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe12c0a2c20 0x7fe12c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:14.679 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.679+0000 7fe143fff700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe12c0a2c20 0x7fe12c0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe144103580 tx=0x7fe134006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:14.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.682+0000 7fe1417fa700 1 -- 172.21.15.179:0/3924901306 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe13801f020 con 0x7fe144103720 2024-06-03T01:03:14.887 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:14.887+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe1440628d0 con 0x7fe12c0a2c20 2024-06-03T01:03:15.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:15 smithi094 ceph-mon[32119]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:15.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:03:15.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:03:15.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:15 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:15.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:15 smithi094 ceph-mon[32119]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T01:03:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:15 smithi179 ceph-mon[37041]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:03:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:03:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:15 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:15 smithi179 ceph-mon[37041]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T01:03:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:15 smithi107 ceph-mon[36622]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-03T01:03:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-03T01:03:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:15 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:15 smithi107 ceph-mon[36622]: osdmap e76: 11 total, 11 up, 11 in 2024-06-03T01:03:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:16 smithi094 ceph-mon[32119]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:03:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:16 smithi094 ceph-mon[32119]: osdmap e77: 11 total, 11 up, 11 in 2024-06-03T01:03:16.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:16 smithi179 ceph-mon[37041]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:03:16.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:16 smithi179 ceph-mon[37041]: osdmap e77: 11 total, 11 up, 11 in 2024-06-03T01:03:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:16 smithi107 ceph-mon[36622]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:03:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:16 smithi107 ceph-mon[36622]: osdmap e77: 11 total, 11 up, 11 in 2024-06-03T01:03:17.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:17 smithi094 ceph-mon[32119]: pgmap v268: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:03:17.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:17 smithi179 ceph-mon[37041]: pgmap v268: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:03:17.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:17 smithi107 ceph-mon[36622]: pgmap v268: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-03T01:03:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:18 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/2244015528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]: dispatch 2024-06-03T01:03:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:18 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]: dispatch 2024-06-03T01:03:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:18 smithi094 ceph-mon[32119]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]': finished 2024-06-03T01:03:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:18 smithi094 ceph-mon[32119]: osdmap e78: 12 total, 11 up, 12 in 2024-06-03T01:03:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:18 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:18 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/2244015528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]: dispatch 2024-06-03T01:03:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:18 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]: dispatch 2024-06-03T01:03:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:18 smithi179 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]': finished 2024-06-03T01:03:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:18 smithi179 ceph-mon[37041]: osdmap e78: 12 total, 11 up, 12 in 2024-06-03T01:03:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:18 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:18 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/2244015528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]: dispatch 2024-06-03T01:03:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:18 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]: dispatch 2024-06-03T01:03:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:18 smithi107 ceph-mon[36622]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5ee1e38-10d2-4513-9d5b-29816f8d6eb5"}]': finished 2024-06-03T01:03:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:18 smithi107 ceph-mon[36622]: osdmap e78: 12 total, 11 up, 12 in 2024-06-03T01:03:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:18 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:19.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:19 smithi094 ceph-mon[32119]: pgmap v270: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-06-03T01:03:19.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:19 smithi094 ceph-mon[32119]: from='client.? 172.21.15.179:0/758332847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:03:19.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:19 smithi179 ceph-mon[37041]: pgmap v270: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-06-03T01:03:19.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:19 smithi179 ceph-mon[37041]: from='client.? 172.21.15.179:0/758332847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:03:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:19 smithi107 ceph-mon[36622]: pgmap v270: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-06-03T01:03:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:19 smithi107 ceph-mon[36622]: from='client.? 172.21.15.179:0/758332847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-03T01:03:21.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:21 smithi094 ceph-mon[32119]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:03:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:21 smithi179 ceph-mon[37041]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:03:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:21 smithi107 ceph-mon[36622]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-03T01:03:23.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:23 smithi094 ceph-mon[32119]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:23 smithi179 ceph-mon[37041]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:23.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:23 smithi107 ceph-mon[36622]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:25.585 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:25 smithi107 ceph-mon[36622]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:25 smithi094 ceph-mon[32119]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:25.604 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:25 smithi179 ceph-mon[37041]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T01:03:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:26 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:26 smithi094 ceph-mon[32119]: Deploying daemon osd.11 on smithi179 2024-06-03T01:03:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T01:03:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:26 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:26 smithi179 ceph-mon[37041]: Deploying daemon osd.11 on smithi179 2024-06-03T01:03:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-03T01:03:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:26 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:26 smithi107 ceph-mon[36622]: Deploying daemon osd.11 on smithi179 2024-06-03T01:03:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:27 smithi094 ceph-mon[32119]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:27 smithi179 ceph-mon[37041]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:27 smithi107 ceph-mon[36622]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:29.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:29 smithi107 ceph-mon[36622]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:29.290 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:29 smithi179 ceph-mon[37041]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:29.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:29 smithi094 ceph-mon[32119]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:31 smithi094 ceph-mon[32119]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:31 smithi179 ceph-mon[37041]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:31 smithi107 ceph-mon[36622]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:33.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:33 smithi179 ceph-mon[37041]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:33 smithi094 ceph-mon[32119]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:33.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:33 smithi107 ceph-mon[36622]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:34.877 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.877+0000 7fe1417fa700 1 -- 172.21.15.179:0/3924901306 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe1440628d0 con 0x7fe12c0a2c20 2024-06-03T01:03:34.877 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 11 on host 'smithi179' 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe12c0a2c20 msgr2=0x7fe12c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe12c0a2c20 0x7fe12c0a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe144103580 tx=0x7fe134006d20 comp rx=0 tx=0).stop 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe144103720 msgr2=0x7fe144198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe144103720 0x7fe144198d70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b560 tx=0x7fe13800b920 comp rx=0 tx=0).stop 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 shutdown_connections 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1441040c0 0x7fe14419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe12c0a2c20 0x7fe12c0a50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe144102520 0x7fe144198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 --2- 172.21.15.179:0/3924901306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe144103720 0x7fe144198d70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 >> 172.21.15.179:0/3924901306 conn(0x7fe1440fdae0 msgr2=0x7fe1441069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 shutdown_connections 2024-06-03T01:03:34.880 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:34.880+0000 7fe14a335700 1 -- 172.21.15.179:0/3924901306 wait complete. 2024-06-03T01:03:35.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:35.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:35.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T01:03:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:35 smithi179 ceph-mon[37041]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T01:03:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:35.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:35.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T01:03:35.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:35 smithi107 ceph-mon[36622]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T01:03:35.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T01:03:35.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:35 smithi094 ceph-mon[32119]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-03T01:03:35.584 DEBUG:teuthology.orchestra.run.smithi179:osd.11> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.11.service 2024-06-03T01:03:35.587 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-03T01:03:35.587 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-03T01:03:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:03:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:36 smithi179 ceph-mon[37041]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T01:03:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:36 smithi179 ceph-mon[37041]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T01:03:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:36 smithi179 ceph-mon[37041]: from='osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:03:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:36 smithi179 ceph-mon[37041]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:03:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:36 smithi179 ceph-mon[37041]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:37.195 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:36 smithi094 ceph-mon[32119]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T01:03:37.195 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:36 smithi094 ceph-mon[32119]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T01:03:37.195 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:37.195 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:36 smithi094 ceph-mon[32119]: from='osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:03:37.195 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:36 smithi094 ceph-mon[32119]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:03:37.195 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:36 smithi094 ceph-mon[32119]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:37.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:36 smithi107 ceph-mon[36622]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-03T01:03:37.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:36 smithi107 ceph-mon[36622]: osdmap e79: 12 total, 11 up, 12 in 2024-06-03T01:03:37.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:36 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:37.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:36 smithi107 ceph-mon[36622]: from='osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:03:37.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:36 smithi107 ceph-mon[36622]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-06-03T01:03:37.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:36 smithi107 ceph-mon[36622]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:37.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:03:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:03:37.615+0000 7fae95fbd700 -1 osd.11 0 waiting for initial osdmap 2024-06-03T01:03:37.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:03:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:03:37.631+0000 7fae8dda8700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-03T01:03:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:37 smithi094 ceph-mon[32119]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:03:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:37 smithi094 ceph-mon[32119]: osdmap e80: 12 total, 11 up, 12 in 2024-06-03T01:03:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:37 smithi094 ceph-mon[32119]: from='osd.11 ' entity='osd.11' 2024-06-03T01:03:38.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:37 smithi107 ceph-mon[36622]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:03:38.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:37 smithi107 ceph-mon[36622]: osdmap e80: 12 total, 11 up, 12 in 2024-06-03T01:03:38.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:38.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:38.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:37 smithi107 ceph-mon[36622]: from='osd.11 ' entity='osd.11' 2024-06-03T01:03:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:37 smithi179 ceph-mon[37041]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-06-03T01:03:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:37 smithi179 ceph-mon[37041]: osdmap e80: 12 total, 11 up, 12 in 2024-06-03T01:03:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:37 smithi179 ceph-mon[37041]: from='osd.11 ' entity='osd.11' 2024-06-03T01:03:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 -- 172.21.15.94:0/434174878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 msgr2=0x7f47cc106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:38.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 --2- 172.21.15.94:0/434174878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 0x7f47cc106450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b0a0 tx=0x7f47c400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 -- 172.21.15.94:0/434174878 shutdown_connections 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 --2- 172.21.15.94:0/434174878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47cc100c90 0x7f47cc1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 --2- 172.21.15.94:0/434174878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 0x7f47cc106450 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 --2- 172.21.15.94:0/434174878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc103600 0x7f47cc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 -- 172.21.15.94:0/434174878 >> 172.21.15.94:0/434174878 conn(0x7f47cc0fc8b0 msgr2=0x7f47cc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 -- 172.21.15.94:0/434174878 shutdown_connections 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.381+0000 7f47d2c00700 1 -- 172.21.15.94:0/434174878 wait complete. 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.382+0000 7f47d2c00700 1 Processor -- start 2024-06-03T01:03:38.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.382+0000 7f47d2c00700 1 -- start start 2024-06-03T01:03:38.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d2c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc100c90 0x7f47cc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:38.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d2c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47cc103600 0x7f47cc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:38.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d2c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 0x7f47cc1a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:38.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d2c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc1145d0 con 0x7f47cc105fd0 2024-06-03T01:03:38.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d2c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc114450 con 0x7f47cc103600 2024-06-03T01:03:38.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d2c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc114750 con 0x7f47cc100c90 2024-06-03T01:03:38.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d1800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc100c90 0x7f47cc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d1800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc100c90 0x7f47cc19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:47572/0 (socket says 172.21.15.94:47572) 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d1800700 1 -- 172.21.15.94:0/516021286 learned_addr learned my addr 172.21.15.94:0/516021286 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d2200700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 0x7f47cc1a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.383+0000 7f47d0e00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47cc103600 0x7f47cc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d1800700 1 -- 172.21.15.94:0/516021286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47cc103600 msgr2=0x7f47cc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d1800700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47cc103600 0x7f47cc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d1800700 1 -- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 msgr2=0x7f47cc1a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d1800700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 0x7f47cc1a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d1800700 1 -- 172.21.15.94:0/516021286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc1a67e0 con 0x7f47cc100c90 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d2200700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 0x7f47cc1a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:03:38.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d1800700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc100c90 0x7f47cc19eb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e9d0 tx=0x7f47c000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47ca000700 1 -- 172.21.15.94:0/516021286 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c000cbd0 con 0x7f47cc100c90 2024-06-03T01:03:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.384+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc1a6ad0 con 0x7f47cc100c90 2024-06-03T01:03:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.385+0000 7f47ca000700 1 -- 172.21.15.94:0/516021286 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c000cd70 con 0x7f47cc100c90 2024-06-03T01:03:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.385+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc1a7060 con 0x7f47cc100c90 2024-06-03T01:03:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.385+0000 7f47ca000700 1 -- 172.21.15.94:0/516021286 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0018680 con 0x7f47cc100c90 2024-06-03T01:03:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.386+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc069180 con 0x7f47cc100c90 2024-06-03T01:03:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.388+0000 7f47ca000700 1 -- 172.21.15.94:0/516021286 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f47c0010430 con 0x7f47cc100c90 2024-06-03T01:03:38.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.389+0000 7f47ca000700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f47b40a2ca0 0x7f47b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:38.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.389+0000 7f47ca000700 1 -- 172.21.15.94:0/516021286 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f47c00ced00 con 0x7f47cc100c90 2024-06-03T01:03:38.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.389+0000 7f47d0e00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f47b40a2ca0 0x7f47b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:38.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.390+0000 7f47d0e00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f47b40a2ca0 0x7f47b40a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009c60 tx=0x7f47bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:38.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.393+0000 7f47ca000700 1 -- 172.21.15.94:0/516021286 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f47c009c180 con 0x7f47cc100c90 2024-06-03T01:03:38.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.646+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f47cc0630a0 con 0x7f47cc100c90 2024-06-03T01:03:38.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.647+0000 7f47ca000700 1 -- 172.21.15.94:0/516021286 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f47c009ba80 con 0x7f47cc100c90 2024-06-03T01:03:38.649 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:03:38.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f47b40a2ca0 msgr2=0x7f47b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:38.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f47b40a2ca0 0x7f47b40a5160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009c60 tx=0x7f47bc009260 comp rx=0 tx=0).stop 2024-06-03T01:03:38.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc100c90 msgr2=0x7f47cc19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc100c90 0x7f47cc19eb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e9d0 tx=0x7f47c000ece0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 shutdown_connections 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f47b40a2ca0 0x7f47b40a5160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47cc105fd0 0x7f47cc1a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47cc100c90 0x7f47cc19eb80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 --2- 172.21.15.94:0/516021286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47cc103600 0x7f47cc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 >> 172.21.15.94:0/516021286 conn(0x7f47cc0fc8b0 msgr2=0x7f47cc101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 shutdown_connections 2024-06-03T01:03:38.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:38.650+0000 7f47d2c00700 1 -- 172.21.15.94:0/516021286 wait complete. 2024-06-03T01:03:39.091 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: purged_snaps scrub starts 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: purged_snaps scrub ok 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] boot 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: osdmap e81: 12 total, 12 up, 12 in 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/516021286' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T01:03:39.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:38 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:39.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: purged_snaps scrub starts 2024-06-03T01:03:39.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: purged_snaps scrub ok 2024-06-03T01:03:39.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:39.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:39.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] boot 2024-06-03T01:03:39.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: osdmap e81: 12 total, 12 up, 12 in 2024-06-03T01:03:39.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:39.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/516021286' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T01:03:39.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:38 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: purged_snaps scrub starts 2024-06-03T01:03:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: purged_snaps scrub ok 2024-06-03T01:03:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-06-03T01:03:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: osd.11 [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] boot 2024-06-03T01:03:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: osdmap e81: 12 total, 12 up, 12 in 2024-06-03T01:03:39.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:03:39.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/516021286' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-03T01:03:39.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:38 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:39.526 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1717376618,"num_in_osds":12,"osd_in_since":1717376597,"num_remapped_pgs":0} 2024-06-03T01:03:39.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:03:39.941 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:03:40.084 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: Detected new or changed devices on smithi179 2024-06-03T01:03:40.084 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.084 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi179 to 4272M 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.085 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:39 smithi094 ceph-mon[32119]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T01:03:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: Detected new or changed devices on smithi179 2024-06-03T01:03:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi179 to 4272M 2024-06-03T01:03:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:39 smithi107 ceph-mon[36622]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T01:03:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: Detected new or changed devices on smithi179 2024-06-03T01:03:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:03:40.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi179 to 4272M 2024-06-03T01:03:40.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:40.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:40.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:40.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:39 smithi179 ceph-mon[37041]: osdmap e82: 12 total, 12 up, 12 in 2024-06-03T01:03:41.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:40 smithi107 ceph-mon[36622]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:41.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:40 smithi094 ceph-mon[32119]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:41.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:40 smithi179 ceph-mon[37041]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.288+0000 7f44bac00700 1 -- 172.21.15.94:0/4267849313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b4074fe0 msgr2=0x7f44b410e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.288+0000 7f44bac00700 1 --2- 172.21.15.94:0/4267849313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b4074fe0 0x7f44b410e950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009990 tx=0x7f44a0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.288+0000 7f44bac00700 1 -- 172.21.15.94:0/4267849313 shutdown_connections 2024-06-03T01:03:42.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.288+0000 7f44bac00700 1 --2- 172.21.15.94:0/4267849313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b410ee90 0x7f44b4111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.288+0000 7f44bac00700 1 --2- 172.21.15.94:0/4267849313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44b40746c0 0x7f44b4074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.288+0000 7f44bac00700 1 --2- 172.21.15.94:0/4267849313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b4074fe0 0x7f44b410e950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.288+0000 7f44bac00700 1 -- 172.21.15.94:0/4267849313 >> 172.21.15.94:0/4267849313 conn(0x7f44b40795f0 msgr2=0x7f44b4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.289+0000 7f44bac00700 1 -- 172.21.15.94:0/4267849313 shutdown_connections 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.289+0000 7f44bac00700 1 -- 172.21.15.94:0/4267849313 wait complete. 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.290+0000 7f44bac00700 1 Processor -- start 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44bac00700 1 -- start start 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44bac00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b40746c0 0x7f44b419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44bac00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44b4074fe0 0x7f44b419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44bac00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b410ee90 0x7f44b419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44bac00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4114a80 con 0x7f44b4074fe0 2024-06-03T01:03:42.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44bac00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4114900 con 0x7f44b40746c0 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44bac00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44b4114c00 con 0x7f44b410ee90 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.291+0000 7f44b3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b40746c0 0x7f44b419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b40746c0 0x7f44b419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:38012/0 (socket says 172.21.15.94:38012) 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3e00700 1 -- 172.21.15.94:0/4166073097 learned_addr learned my addr 172.21.15.94:0/4166073097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b8c00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b410ee90 0x7f44b419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3400700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44b4074fe0 0x7f44b419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3e00700 1 -- 172.21.15.94:0/4166073097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b410ee90 msgr2=0x7f44b419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3e00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b410ee90 0x7f44b419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3e00700 1 -- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44b4074fe0 msgr2=0x7f44b419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3e00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44b4074fe0 0x7f44b419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3e00700 1 -- 172.21.15.94:0/4166073097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b419fe60 con 0x7f44b40746c0 2024-06-03T01:03:42.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.292+0000 7f44b3400700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44b4074fe0 0x7f44b419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:03:42.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.293+0000 7f44b3e00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b40746c0 0x7f44b419ae50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44a400eb30 tx=0x7f44a400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:42.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.293+0000 7f44b0c00700 1 -- 172.21.15.94:0/4166073097 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a400cc90 con 0x7f44b40746c0 2024-06-03T01:03:42.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.293+0000 7f44b0c00700 1 -- 172.21.15.94:0/4166073097 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4004500 con 0x7f44b40746c0 2024-06-03T01:03:42.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.293+0000 7f44b0c00700 1 -- 172.21.15.94:0/4166073097 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4010430 con 0x7f44b40746c0 2024-06-03T01:03:42.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.293+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b41a0150 con 0x7f44b40746c0 2024-06-03T01:03:42.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.293+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b41a79d0 con 0x7f44b40746c0 2024-06-03T01:03:42.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.294+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b4069180 con 0x7f44b40746c0 2024-06-03T01:03:42.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.296+0000 7f44b0c00700 1 -- 172.21.15.94:0/4166073097 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f44a4003f70 con 0x7f44b40746c0 2024-06-03T01:03:42.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.298+0000 7f44b0c00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f44980a2ca0 0x7f44980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:42.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.298+0000 7f44b0c00700 1 -- 172.21.15.94:0/4166073097 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f44a40cefd0 con 0x7f44b40746c0 2024-06-03T01:03:42.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.298+0000 7f44b3400700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f44980a2ca0 0x7f44980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.299+0000 7f44b3400700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f44980a2ca0 0x7f44980a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44b4110b90 tx=0x7f44a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.300+0000 7f44b0c00700 1 -- 172.21.15.94:0/4166073097 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f44a409c3c0 con 0x7f44b40746c0 2024-06-03T01:03:42.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.522+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44b419c1b0 con 0x7f44b40746c0 2024-06-03T01:03:42.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.524+0000 7f44b0c00700 1 -- 172.21.15.94:0/4166073097 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18455 (secure 0 0 0) 0x7f44a409bcc0 con 0x7f44b40746c0 2024-06-03T01:03:42.525 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:03:42.525 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:03:39.621420+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:03:42.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f44980a2ca0 msgr2=0x7f44980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f44980a2ca0 0x7f44980a5160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44b4110b90 tx=0x7f44a001a040 comp rx=0 tx=0).stop 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b40746c0 msgr2=0x7f44b419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b40746c0 0x7f44b419ae50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44a400eb30 tx=0x7f44a400ee40 comp rx=0 tx=0).stop 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 shutdown_connections 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44b40746c0 0x7f44b419ae50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f44980a2ca0 0x7f44980a5160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b410ee90 0x7f44b419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 --2- 172.21.15.94:0/4166073097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44b4074fe0 0x7f44b419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:42.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.527+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 >> 172.21.15.94:0/4166073097 conn(0x7f44b40795f0 msgr2=0x7f44b40ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:42.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.528+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 shutdown_connections 2024-06-03T01:03:42.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:42.528+0000 7f44bac00700 1 -- 172.21.15.94:0/4166073097 wait complete. 2024-06-03T01:03:43.398 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T00:59:20.779028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-03T01:03:43.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-03T01:03:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:43 smithi094 ceph-mon[32119]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:43 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/4166073097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:03:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:43 smithi179 ceph-mon[37041]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:43 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/4166073097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:03:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:43 smithi107 ceph-mon[36622]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:43 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/4166073097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:03:43.838 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:03:45.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:45 smithi094 ceph-mon[32119]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:45 smithi179 ceph-mon[37041]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:45.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:45 smithi107 ceph-mon[36622]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 -- 172.21.15.94:0/3162236149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100a80 msgr2=0x7f3104100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 --2- 172.21.15.94:0/3162236149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100a80 0x7f3104100f00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30f00098f0 tx=0x7f30f0009c00 comp rx=0 tx=0).stop 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 -- 172.21.15.94:0/3162236149 shutdown_connections 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 --2- 172.21.15.94:0/3162236149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 0x7f3104111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 --2- 172.21.15.94:0/3162236149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31041033a0 0x7f3104103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 --2- 172.21.15.94:0/3162236149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100a80 0x7f3104100f00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 -- 172.21.15.94:0/3162236149 >> 172.21.15.94:0/3162236149 conn(0x7f3104077a00 msgr2=0x7f31040fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 -- 172.21.15.94:0/3162236149 shutdown_connections 2024-06-03T01:03:46.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.174+0000 7f3109c00700 1 -- 172.21.15.94:0/3162236149 wait complete. 2024-06-03T01:03:46.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.175+0000 7f3109c00700 1 Processor -- start 2024-06-03T01:03:46.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3109c00700 1 -- start start 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3109c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3104100a80 0x7f310419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3109c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 0x7f310419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3109c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31041033a0 0x7f310419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3109c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f31041149d0 con 0x7f3104101520 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3109c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3104114850 con 0x7f31041033a0 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3109c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3104114b50 con 0x7f3104100a80 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3102000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 0x7f310419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3102a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3104100a80 0x7f310419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3102000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 0x7f310419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45704/0 (socket says 172.21.15.94:45704) 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3102000700 1 -- 172.21.15.94:0/1526249827 learned_addr learned my addr 172.21.15.94:0/1526249827 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:03:46.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.176+0000 7f3103400700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31041033a0 0x7f310419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:46.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3102000700 1 -- 172.21.15.94:0/1526249827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3104100a80 msgr2=0x7f310419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:46.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3102000700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3104100a80 0x7f310419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3102000700 1 -- 172.21.15.94:0/1526249827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31041033a0 msgr2=0x7f310419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3102000700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31041033a0 0x7f310419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3102000700 1 -- 172.21.15.94:0/1526249827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310419fed0 con 0x7f3104101520 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3103400700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31041033a0 0x7f310419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3102a00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3104100a80 0x7f310419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.177+0000 7f3102000700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 0x7f310419b400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005950 tx=0x7f30f000fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.178+0000 7f30fb400700 1 -- 172.21.15.94:0/1526249827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f00051f0 con 0x7f3104101520 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.178+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31041a0160 con 0x7f3104101520 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.178+0000 7f30fb400700 1 -- 172.21.15.94:0/1526249827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f000b560 con 0x7f3104101520 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.178+0000 7f30fb400700 1 -- 172.21.15.94:0/1526249827 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0003a40 con 0x7f3104101520 2024-06-03T01:03:46.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.178+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041a7a40 con 0x7f3104101520 2024-06-03T01:03:46.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.179+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3104069180 con 0x7f3104101520 2024-06-03T01:03:46.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.181+0000 7f30fb400700 1 -- 172.21.15.94:0/1526249827 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f30f000ba80 con 0x7f3104101520 2024-06-03T01:03:46.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.183+0000 7f30fb400700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f30e80a2ca0 0x7f30e80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:46.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.183+0000 7f30fb400700 1 -- 172.21.15.94:0/1526249827 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f30f00cfcf0 con 0x7f3104101520 2024-06-03T01:03:46.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.183+0000 7f3102a00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f30e80a2ca0 0x7f30e80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:46.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.184+0000 7f3102a00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f30e80a2ca0 0x7f30e80a5160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005fd0 tx=0x7f30f4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:46.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.185+0000 7f30fb400700 1 -- 172.21.15.94:0/1526249827 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f30f005e850 con 0x7f3104101520 2024-06-03T01:03:46.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.409+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f310419c220 con 0x7f3104101520 2024-06-03T01:03:46.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.410+0000 7f30fb400700 1 -- 172.21.15.94:0/1526249827 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f30f0013030 con 0x7f3104101520 2024-06-03T01:03:46.411 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f30e80a2ca0 msgr2=0x7f30e80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f30e80a2ca0 0x7f30e80a5160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005fd0 tx=0x7f30f4009310 comp rx=0 tx=0).stop 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 msgr2=0x7f310419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 0x7f310419b400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005950 tx=0x7f30f000fb20 comp rx=0 tx=0).stop 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 shutdown_connections 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31041033a0 0x7f310419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f30e80a2ca0 0x7f30e80a5160 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3104100a80 0x7f310419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 --2- 172.21.15.94:0/1526249827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3104101520 0x7f310419b400 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.413+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 >> 172.21.15.94:0/1526249827 conn(0x7f3104077a00 msgr2=0x7f31040fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.414+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 shutdown_connections 2024-06-03T01:03:46.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:46.414+0000 7f3109c00700 1 -- 172.21.15.94:0/1526249827 wait complete. 2024-06-03T01:03:47.139 INFO:tasks.cephadm:Adding mds.a on smithi094 2024-06-03T01:03:47.139 INFO:tasks.cephadm:Adding mds.d on smithi094 2024-06-03T01:03:47.139 INFO:tasks.cephadm:Adding mds.g on smithi094 2024-06-03T01:03:47.139 INFO:tasks.cephadm:Adding mds.j on smithi094 2024-06-03T01:03:47.139 INFO:tasks.cephadm:Adding mds.b on smithi107 2024-06-03T01:03:47.140 INFO:tasks.cephadm:Adding mds.e on smithi107 2024-06-03T01:03:47.140 INFO:tasks.cephadm:Adding mds.h on smithi107 2024-06-03T01:03:47.140 INFO:tasks.cephadm:Adding mds.k on smithi107 2024-06-03T01:03:47.140 INFO:tasks.cephadm:Adding mds.c on smithi179 2024-06-03T01:03:47.140 INFO:tasks.cephadm:Adding mds.f on smithi179 2024-06-03T01:03:47.140 INFO:tasks.cephadm:Adding mds.i on smithi179 2024-06-03T01:03:47.140 INFO:tasks.cephadm:Adding mds.l on smithi179 2024-06-03T01:03:47.141 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l' 2024-06-03T01:03:47.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:47 smithi094 ceph-mon[32119]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:47.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:47 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1526249827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:03:47.401 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:03:47.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:47 smithi179 ceph-mon[37041]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:47.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:47 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1526249827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:03:47.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:47 smithi107 ceph-mon[36622]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:47.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:47 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1526249827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:03:49.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3625042515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca0102fc0 msgr2=0x7f2ca01053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:49.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3625042515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca0102fc0 0x7f2ca01053b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009990 tx=0x7f2c94009ca0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.118 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3625042515 shutdown_connections 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3625042515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ca0108220 0x7f2ca010a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3625042515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca0102fc0 0x7f2ca01053b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3625042515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca01058f0 0x7f2ca0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3625042515 >> 172.21.15.179:0/3625042515 conn(0x7f2ca00fc9f0 msgr2=0x7f2ca00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3625042515 shutdown_connections 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.116+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3625042515 wait complete. 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 Processor -- start 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 -- start start 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ca0102fc0 0x7f2ca01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca01058f0 0x7f2ca0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0108220 0x7f2ca019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca010e100 con 0x7f2ca0108220 2024-06-03T01:03:49.119 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca010df80 con 0x7f2ca0102fc0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca6a66700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca010e280 con 0x7f2ca01058f0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2c9ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca01058f0 0x7f2ca0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2c9ffff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca01058f0 0x7f2ca0198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33680/0 (socket says 172.21.15.179:33680) 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2c9ffff700 1 -- 172.21.15.179:0/3478730870 learned_addr learned my addr 172.21.15.179:0/3478730870 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.117+0000 7f2ca4802700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ca0102fc0 0x7f2ca01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9ffff700 1 -- 172.21.15.179:0/3478730870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ca0102fc0 msgr2=0x7f2ca01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9ffff700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ca0102fc0 0x7f2ca01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9ffff700 1 -- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0108220 msgr2=0x7f2ca019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9ffff700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0108220 0x7f2ca019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9ffff700 1 -- 172.21.15.179:0/3478730870 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca01a0340 con 0x7f2ca01058f0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9ffff700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca01058f0 0x7f2ca0198c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c950 tx=0x7f2c8c00cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9dffb700 1 -- 172.21.15.179:0/3478730870 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c007ac0 con 0x7f2ca01058f0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9dffb700 1 -- 172.21.15.179:0/3478730870 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8c004d10 con 0x7f2ca01058f0 2024-06-03T01:03:49.120 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca01a0630 con 0x7f2ca01058f0 2024-06-03T01:03:49.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2c9dffb700 1 -- 172.21.15.179:0/3478730870 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00f450 con 0x7f2ca01058f0 2024-06-03T01:03:49.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.118+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca01a0b70 con 0x7f2ca01058f0 2024-06-03T01:03:49.121 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.119+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca0069180 con 0x7f2ca01058f0 2024-06-03T01:03:49.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.120+0000 7f2c9dffb700 1 -- 172.21.15.179:0/3478730870 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c8c004750 con 0x7f2ca01058f0 2024-06-03T01:03:49.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.121+0000 7f2c9dffb700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2c900a2c20 0x7f2c900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:49.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.121+0000 7f2c9dffb700 1 -- 172.21.15.179:0/3478730870 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2c8c0ceaa0 con 0x7f2ca01058f0 2024-06-03T01:03:49.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.121+0000 7f2ca4802700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2c900a2c20 0x7f2c900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:49.122 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.122+0000 7f2ca4802700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2c900a2c20 0x7f2c900a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400b5c0 tx=0x7f2c94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:49.125 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.125+0000 7f2c9dffb700 1 -- 172.21.15.179:0/3478730870 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2c8c014030 con 0x7f2ca01058f0 2024-06-03T01:03:49.334 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.334+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}) -- 0x7f2ca00629d0 con 0x7f2c900a2c20 2024-06-03T01:03:49.350 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.351+0000 7f2c9dffb700 1 -- 172.21.15.179:0/3478730870 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2ca00629d0 con 0x7f2c900a2c20 2024-06-03T01:03:49.350 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mds.all update... 2024-06-03T01:03:49.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2c900a2c20 msgr2=0x7f2c900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:49.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2c900a2c20 0x7f2c900a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400b5c0 tx=0x7f2c94005fd0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca01058f0 msgr2=0x7f2ca0198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:49.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca01058f0 0x7f2ca0198c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c950 tx=0x7f2c8c00cc60 comp rx=0 tx=0).stop 2024-06-03T01:03:49.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 shutdown_connections 2024-06-03T01:03:49.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ca0102fc0 0x7f2ca01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2c900a2c20 0x7f2c900a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ca0108220 0x7f2ca019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 --2- 172.21.15.179:0/3478730870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ca01058f0 0x7f2ca0198c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:49.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 >> 172.21.15.179:0/3478730870 conn(0x7f2ca00fc9f0 msgr2=0x7f2ca0103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:49.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 shutdown_connections 2024-06-03T01:03:49.354 INFO:teuthology.orchestra.run.smithi179.stderr:2024-06-03T01:03:49.353+0000 7f2ca6a66700 1 -- 172.21.15.179:0/3478730870 wait complete. 2024-06-03T01:03:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:49 smithi094 ceph-mon[32119]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:49 smithi179 ceph-mon[37041]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:49.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:49 smithi107 ceph-mon[36622]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:50.001 DEBUG:teuthology.orchestra.run.smithi094:mds.a> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.a.service 2024-06-03T01:03:50.006 DEBUG:teuthology.orchestra.run.smithi094:mds.d> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.d.service 2024-06-03T01:03:50.051 DEBUG:teuthology.orchestra.run.smithi094:mds.g> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.g.service 2024-06-03T01:03:50.098 DEBUG:teuthology.orchestra.run.smithi094:mds.j> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.j.service 2024-06-03T01:03:50.143 DEBUG:teuthology.orchestra.run.smithi107:mds.b> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.b.service 2024-06-03T01:03:50.146 DEBUG:teuthology.orchestra.run.smithi107:mds.e> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.e.service 2024-06-03T01:03:50.190 DEBUG:teuthology.orchestra.run.smithi107:mds.h> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.h.service 2024-06-03T01:03:50.233 DEBUG:teuthology.orchestra.run.smithi107:mds.k> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.k.service 2024-06-03T01:03:50.276 DEBUG:teuthology.orchestra.run.smithi179:mds.c> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.c.service 2024-06-03T01:03:50.279 DEBUG:teuthology.orchestra.run.smithi179:mds.f> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.f.service 2024-06-03T01:03:50.321 DEBUG:teuthology.orchestra.run.smithi179:mds.i> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.i.service 2024-06-03T01:03:50.364 DEBUG:teuthology.orchestra.run.smithi179:mds.l> sudo journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.l.service 2024-06-03T01:03:50.407 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-03T01:03:50.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:03:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:50 smithi107 ceph-mon[36622]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:03:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:50 smithi107 ceph-mon[36622]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-06-03T01:03:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:50 smithi107 ceph-mon[36622]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:50 smithi179 ceph-mon[37041]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:03:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:50 smithi179 ceph-mon[37041]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-06-03T01:03:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:50 smithi179 ceph-mon[37041]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:50.839 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:03:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:50 smithi094 ceph-mon[32119]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:03:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:50 smithi094 ceph-mon[32119]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-06-03T01:03:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:03:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:50 smithi094 ceph-mon[32119]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:51.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:51.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:51.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:51.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:51.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:51.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:51 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:51.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:51 smithi094 ceph-mon[32119]: Deploying daemon mds.l on smithi179 2024-06-03T01:03:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:51 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:51 smithi179 ceph-mon[37041]: Deploying daemon mds.l on smithi179 2024-06-03T01:03:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:03:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:51 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:51 smithi107 ceph-mon[36622]: Deploying daemon mds.l on smithi179 2024-06-03T01:03:52.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:52 smithi094 ceph-mon[32119]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:52.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:52 smithi179 ceph-mon[37041]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:52 smithi107 ceph-mon[36622]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 -- 172.21.15.94:0/413028717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0102740 msgr2=0x7f6bc0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/413028717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0102740 0x7f6bc0102b20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009920 tx=0x7f6bb0009c30 comp rx=0 tx=0).stop 2024-06-03T01:03:53.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 -- 172.21.15.94:0/413028717 shutdown_connections 2024-06-03T01:03:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/413028717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc0107840 0x7f6bc010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/413028717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0102740 0x7f6bc0102b20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/413028717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc01030f0 0x7f6bc0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 -- 172.21.15.94:0/413028717 >> 172.21.15.94:0/413028717 conn(0x7f6bc00795f0 msgr2=0x7f6bc0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 -- 172.21.15.94:0/413028717 shutdown_connections 2024-06-03T01:03:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.122+0000 7f6bc4c00700 1 -- 172.21.15.94:0/413028717 wait complete. 2024-06-03T01:03:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.123+0000 7f6bc4c00700 1 Processor -- start 2024-06-03T01:03:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.123+0000 7f6bc4c00700 1 -- start start 2024-06-03T01:03:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.123+0000 7f6bc4c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc0102740 0x7f6bc019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.123+0000 7f6bc4c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc01030f0 0x7f6bc019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bc4c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc0107840 0x7f6bc019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bc4c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0112760 con 0x7f6bc01030f0 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bc4c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01125e0 con 0x7f6bc0107840 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bc4c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01128e0 con 0x7f6bc0102740 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbde00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc0102740 0x7f6bc019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc01030f0 0x7f6bc019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc01030f0 0x7f6bc019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45714/0 (socket says 172.21.15.94:45714) 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbde00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc0102740 0x7f6bc019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38958/0 (socket says 172.21.15.94:38958) 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 -- 172.21.15.94:0/2590105131 learned_addr learned my addr 172.21.15.94:0/2590105131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbe800700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc0107840 0x7f6bc019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 -- 172.21.15.94:0/2590105131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc0102740 msgr2=0x7f6bc019aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc0102740 0x7f6bc019aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 -- 172.21.15.94:0/2590105131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc0107840 msgr2=0x7f6bc019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc0107840 0x7f6bc019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.124+0000 7f6bbd400700 1 -- 172.21.15.94:0/2590105131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc019feb0 con 0x7f6bc01030f0 2024-06-03T01:03:53.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.125+0000 7f6bbd400700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc01030f0 0x7f6bc019b430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00eb30 tx=0x7f6bac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:53.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.125+0000 7f6bb6a00700 1 -- 172.21.15.94:0/2590105131 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00cbd0 con 0x7f6bc01030f0 2024-06-03T01:03:53.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.125+0000 7f6bb6a00700 1 -- 172.21.15.94:0/2590105131 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bac004500 con 0x7f6bc01030f0 2024-06-03T01:03:53.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.125+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc01a01a0 con 0x7f6bc01030f0 2024-06-03T01:03:53.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.125+0000 7f6bb6a00700 1 -- 172.21.15.94:0/2590105131 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac010430 con 0x7f6bc01030f0 2024-06-03T01:03:53.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.125+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc01a7a20 con 0x7f6bc01030f0 2024-06-03T01:03:53.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.126+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc0069180 con 0x7f6bc01030f0 2024-06-03T01:03:53.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.129+0000 7f6bb6a00700 1 -- 172.21.15.94:0/2590105131 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6bac003f00 con 0x7f6bc01030f0 2024-06-03T01:03:53.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.131+0000 7f6bb6a00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ba40a2ca0 0x7f6ba40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.131+0000 7f6bb6a00700 1 -- 172.21.15.94:0/2590105131 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6bac0cefe0 con 0x7f6bc01030f0 2024-06-03T01:03:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.131+0000 7f6bbde00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ba40a2ca0 0x7f6ba40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:53.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.132+0000 7f6bbde00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ba40a2ca0 0x7f6ba40a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c010 tx=0x7f6bb0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:53.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.133+0000 7f6bb6a00700 1 -- 172.21.15.94:0/2590105131 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6bac01f340 con 0x7f6bc01030f0 2024-06-03T01:03:53.355 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:03:53 smithi179 systemd[1]: Started Ceph mds.l for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:03:53.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.425+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bc019c200 con 0x7f6bc01030f0 2024-06-03T01:03:53.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.426+0000 7f6bb6a00700 1 -- 172.21.15.94:0/2590105131 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f6bc019c200 con 0x7f6bc01030f0 2024-06-03T01:03:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:03:53.426 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-03T01:03:53.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ba40a2ca0 msgr2=0x7f6ba40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:53.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ba40a2ca0 0x7f6ba40a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c010 tx=0x7f6bb0019040 comp rx=0 tx=0).stop 2024-06-03T01:03:53.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc01030f0 msgr2=0x7f6bc019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:53.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc01030f0 0x7f6bc019b430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00eb30 tx=0x7f6bac00ee40 comp rx=0 tx=0).stop 2024-06-03T01:03:53.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 shutdown_connections 2024-06-03T01:03:53.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bc0107840 0x7f6bc019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ba40a2ca0 0x7f6ba40a5160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc0102740 0x7f6bc019aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 --2- 172.21.15.94:0/2590105131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc01030f0 0x7f6bc019b430 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:53.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 >> 172.21.15.94:0/2590105131 conn(0x7f6bc00795f0 msgr2=0x7f6bc0100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:53.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 shutdown_connections 2024-06-03T01:03:53.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:53.429+0000 7f6bc4c00700 1 -- 172.21.15.94:0/2590105131 wait complete. 2024-06-03T01:03:53.431 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-06-03T01:03:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6832/2660538909,v1:172.21.15.179:6833/2660538909] up:boot 2024-06-03T01:03:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: fsmap 1 up:standby 2024-06-03T01:03:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:03:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:54.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:54.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:54.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:54 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2590105131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:03:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6832/2660538909,v1:172.21.15.179:6833/2660538909] up:boot 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: fsmap 1 up:standby 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:54 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2590105131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:03:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6832/2660538909,v1:172.21.15.179:6833/2660538909] up:boot 2024-06-03T01:03:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: fsmap 1 up:standby 2024-06-03T01:03:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:03:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:54.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:54.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:54.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:54 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2590105131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:03:55.372 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:55 smithi107 ceph-mon[36622]: Deploying daemon mds.k on smithi107 2024-06-03T01:03:55.373 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:55 smithi107 ceph-mon[36622]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:55 smithi094 ceph-mon[32119]: Deploying daemon mds.k on smithi107 2024-06-03T01:03:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:55 smithi094 ceph-mon[32119]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:55 smithi179 ceph-mon[37041]: Deploying daemon mds.k on smithi107 2024-06-03T01:03:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:55 smithi179 ceph-mon[37041]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:55.754 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:55 smithi107 systemd[1]: Starting Ceph mds.k for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:03:56.033 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:55 smithi107 podman[66068]: 2024-06-03 01:03:55.949784413 +0000 UTC m=+0.094855270 container create fb8529a3db0be1c3ae825ecf39066b007adbe4444b519205e9329b569508bc1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-03T01:03:56.033 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:55 smithi107 podman[66068]: 2024-06-03 01:03:55.872669874 +0000 UTC m=+0.017740719 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:03:56.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:03:56.342 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:56 smithi107 podman[66068]: 2024-06-03 01:03:56.202120587 +0000 UTC m=+0.347191423 container init fb8529a3db0be1c3ae825ecf39066b007adbe4444b519205e9329b569508bc1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-03T01:03:56.343 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:56 smithi107 podman[66068]: 2024-06-03 01:03:56.205003898 +0000 UTC m=+0.350074735 container start fb8529a3db0be1c3ae825ecf39066b007adbe4444b519205e9329b569508bc1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-03T01:03:56.343 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:56 smithi107 ceph-mds[66088]: -- 172.21.15.107:0/1420656680 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556df2057a40 con 0x556df202a800 2024-06-03T01:03:56.343 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:56 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k[66083]: starting mds.k at 2024-06-03T01:03:56.343 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:56 smithi107 bash[66068]: fb8529a3db0be1c3ae825ecf39066b007adbe4444b519205e9329b569508bc1d 2024-06-03T01:03:56.343 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:03:56 smithi107 systemd[1]: Started Ceph mds.k for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:03:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:03:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6832/1330319563,v1:172.21.15.107:6833/1330319563] up:boot 2024-06-03T01:03:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: fsmap 2 up:standby 2024-06-03T01:03:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:03:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:57.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:57.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6832/1330319563,v1:172.21.15.107:6833/1330319563] up:boot 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: fsmap 2 up:standby 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6832/1330319563,v1:172.21.15.107:6833/1330319563] up:boot 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: fsmap 2 up:standby 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:03:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:03:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:03:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:03:58.577 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:58 smithi094 ceph-mon[32119]: Deploying daemon mds.j on smithi094 2024-06-03T01:03:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:58 smithi179 ceph-mon[37041]: Deploying daemon mds.j on smithi094 2024-06-03T01:03:58.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:58 smithi107 ceph-mon[36622]: Deploying daemon mds.j on smithi094 2024-06-03T01:03:59.516 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:03:59 smithi094 ceph-mon[32119]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:03:59 smithi179 ceph-mon[37041]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:59.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 -- 172.21.15.94:0/1742269101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 msgr2=0x7f4e801029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/1742269101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e801029e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e700099c0 tx=0x7f4e70009cd0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 -- 172.21.15.94:0/1742269101 shutdown_connections 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/1742269101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e801076e0 0x7f4e8010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/1742269101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e801029e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/1742269101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e80102fb0 0x7f4e80107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 -- 172.21.15.94:0/1742269101 >> 172.21.15.94:0/1742269101 conn(0x7f4e800fdeb0 msgr2=0x7f4e801002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 -- 172.21.15.94:0/1742269101 shutdown_connections 2024-06-03T01:03:59.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.682+0000 7f4e7fe00700 1 -- 172.21.15.94:0/1742269101 wait complete. 2024-06-03T01:03:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.683+0000 7f4e7fe00700 1 Processor -- start 2024-06-03T01:03:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7fe00700 1 -- start start 2024-06-03T01:03:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e8019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7fe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e80102fb0 0x7f4e8019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e801076e0 0x7f4e801989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7fe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801126c0 con 0x7f4e80102600 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80112540 con 0x7f4e801076e0 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7fe00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80112840 con 0x7f4e80102fb0 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e8019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e8019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41778/0 (socket says 172.21.15.94:41778) 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.684+0000 7f4e7ea00700 1 -- 172.21.15.94:0/214527913 learned_addr learned my addr 172.21.15.94:0/214527913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.685+0000 7f4e7f400700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e801076e0 0x7f4e801989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.685+0000 7f4e7ea00700 1 -- 172.21.15.94:0/214527913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e80102fb0 msgr2=0x7f4e8019ed10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-03T01:03:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.685+0000 7f4e7ea00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e80102fb0 0x7f4e8019ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.685+0000 7f4e7ea00700 1 -- 172.21.15.94:0/214527913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e801076e0 msgr2=0x7f4e801989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.685+0000 7f4e7ea00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e801076e0 0x7f4e801989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.685+0000 7f4e7ea00700 1 -- 172.21.15.94:0/214527913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e80199140 con 0x7f4e80102600 2024-06-03T01:03:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.685+0000 7f4e7ea00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e8019e7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e70006010 tx=0x7f4e70003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.686+0000 7f4e77400700 1 -- 172.21.15.94:0/214527913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70003680 con 0x7f4e80102600 2024-06-03T01:03:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.686+0000 7f4e77400700 1 -- 172.21.15.94:0/214527913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e700055a0 con 0x7f4e80102600 2024-06-03T01:03:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.686+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801993d0 con 0x7f4e80102600 2024-06-03T01:03:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.686+0000 7f4e77400700 1 -- 172.21.15.94:0/214527913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e700173f0 con 0x7f4e80102600 2024-06-03T01:03:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.686+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a56a0 con 0x7f4e80102600 2024-06-03T01:03:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.687+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e80069180 con 0x7f4e80102600 2024-06-03T01:03:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.690+0000 7f4e77400700 1 -- 172.21.15.94:0/214527913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4e7000f680 con 0x7f4e80102600 2024-06-03T01:03:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.692+0000 7f4e77400700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4e640a2e40 0x7f4e640a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:03:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.692+0000 7f4e77400700 1 -- 172.21.15.94:0/214527913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4e700cfde0 con 0x7f4e80102600 2024-06-03T01:03:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.692+0000 7f4e7e000700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4e640a2e40 0x7f4e640a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:03:59.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.693+0000 7f4e7e000700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4e640a2e40 0x7f4e640a5300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e80108c80 tx=0x7f4e6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:03:59.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.695+0000 7f4e77400700 1 -- 172.21.15.94:0/214527913 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e70013070 con 0x7f4e80102600 2024-06-03T01:03:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:03:59 smithi107 ceph-mon[36622]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:03:59.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.964+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e80199ec0 con 0x7f4e80102600 2024-06-03T01:03:59.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.965+0000 7f4e77400700 1 -- 172.21.15.94:0/214527913 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f4e7009d1d0 con 0x7f4e80102600 2024-06-03T01:03:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:03:59.965 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-03T01:03:59.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4e640a2e40 msgr2=0x7f4e640a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:59.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4e640a2e40 0x7f4e640a5300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e80108c80 tx=0x7f4e6c009260 comp rx=0 tx=0).stop 2024-06-03T01:03:59.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 msgr2=0x7f4e8019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:03:59.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e8019e7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4e70006010 tx=0x7f4e70003b40 comp rx=0 tx=0).stop 2024-06-03T01:03:59.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 shutdown_connections 2024-06-03T01:03:59.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e801076e0 0x7f4e801989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4e640a2e40 0x7f4e640a5300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e80102600 0x7f4e8019e7d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 --2- 172.21.15.94:0/214527913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e80102fb0 0x7f4e8019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:03:59.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 >> 172.21.15.94:0/214527913 conn(0x7f4e800fdeb0 msgr2=0x7f4e801001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:03:59.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 shutdown_connections 2024-06-03T01:03:59.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:03:59.968+0000 7f4e7fe00700 1 -- 172.21.15.94:0/214527913 wait complete. 2024-06-03T01:03:59.969 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-06-03T01:04:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:00 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/214527913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:00 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/214527913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:00 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/214527913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:01.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:01 smithi094 ceph-mon[32119]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:01.568 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:04:01 smithi094 podman[72544]: 2024-06-03 01:04:01.408867583 +0000 UTC m=+0.140735125 container create 85d667e7ee530d0aacc6c276c8727b189462c19871b9eb6731c65e4811cbe7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-03T01:04:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:01 smithi179 ceph-mon[37041]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:01.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:01 smithi107 ceph-mon[36622]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:02.099 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:04:01 smithi094 podman[72544]: 2024-06-03 01:04:01.687976601 +0000 UTC m=+0.419844190 container init 85d667e7ee530d0aacc6c276c8727b189462c19871b9eb6731c65e4811cbe7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:04:02.099 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:04:01 smithi094 podman[72544]: 2024-06-03 01:04:01.697921457 +0000 UTC m=+0.429789009 container start 85d667e7ee530d0aacc6c276c8727b189462c19871b9eb6731c65e4811cbe7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:04:02.099 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:04:01 smithi094 ceph-mds[72563]: -- 172.21.15.94:0/614322215 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d99e17a40 con 0x557d99dea800 2024-06-03T01:04:02.100 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:04:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j[72559]: starting mds.j at 2024-06-03T01:04:02.100 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:04:01 smithi094 bash[72544]: 85d667e7ee530d0aacc6c276c8727b189462c19871b9eb6731c65e4811cbe7df 2024-06-03T01:04:02.100 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:04:01 smithi094 systemd[1]: Started Ceph mds.j for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:03.123 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:04:03.172 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.172 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.172 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.172 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:03.172 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:03.172 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:03.172 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: Deploying daemon mds.d on smithi094 2024-06-03T01:04:03.173 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:03.173 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.94:6834/3523136446,v1:172.21.15.94:6835/3523136446] up:boot 2024-06-03T01:04:03.173 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: fsmap 3 up:standby 2024-06-03T01:04:03.173 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:02 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T01:04:03.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:03.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:03.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:03.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: Deploying daemon mds.d on smithi094 2024-06-03T01:04:03.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:03.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.94:6834/3523136446,v1:172.21.15.94:6835/3523136446] up:boot 2024-06-03T01:04:03.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: fsmap 3 up:standby 2024-06-03T01:04:03.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:02 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T01:04:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: Deploying daemon mds.d on smithi094 2024-06-03T01:04:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.94:6834/3523136446,v1:172.21.15.94:6835/3523136446] up:boot 2024-06-03T01:04:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: fsmap 3 up:standby 2024-06-03T01:04:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:02 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T01:04:04.186 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:05.100 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:04 smithi094 systemd[1]: Starting Ceph mds.d for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:05.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:05 smithi094 ceph-mon[32119]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:05 smithi179 ceph-mon[37041]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:05 smithi107 ceph-mon[36622]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:05.850 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:05 smithi094 podman[72940]: 2024-06-03 01:04:05.708672758 +0000 UTC m=+0.034229757 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:05.850 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:05 smithi094 podman[72940]: 2024-06-03 01:04:05.849256442 +0000 UTC m=+0.174813512 container create 95656a1bf6c1de201bc352574d965f7671b2e735712ec1ebcd661aa1da6317d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:04:06.410 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:06 smithi094 podman[72940]: 2024-06-03 01:04:06.356569399 +0000 UTC m=+0.682126411 container init 95656a1bf6c1de201bc352574d965f7671b2e735712ec1ebcd661aa1da6317d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-06-03T01:04:06.410 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:06 smithi094 podman[72940]: 2024-06-03 01:04:06.365471217 +0000 UTC m=+0.691028224 container start 95656a1bf6c1de201bc352574d965f7671b2e735712ec1ebcd661aa1da6317d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:04:06.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.645+0000 7f0922e00700 1 -- 172.21.15.94:0/1397017290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c1046b0 msgr2=0x7f091c104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:06.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.645+0000 7f0922e00700 1 --2- 172.21.15.94:0/1397017290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c1046b0 0x7f091c104b30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0908009990 tx=0x7f0908009ca0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.645+0000 7f0922e00700 1 -- 172.21.15.94:0/1397017290 shutdown_connections 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.645+0000 7f0922e00700 1 --2- 172.21.15.94:0/1397017290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c10a6b0 0x7f091c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.645+0000 7f0922e00700 1 --2- 172.21.15.94:0/1397017290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c105070 0x7f091c111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.645+0000 7f0922e00700 1 --2- 172.21.15.94:0/1397017290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c1046b0 0x7f091c104b30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.645+0000 7f0922e00700 1 -- 172.21.15.94:0/1397017290 >> 172.21.15.94:0/1397017290 conn(0x7f091c1001a0 msgr2=0x7f091c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.646+0000 7f0922e00700 1 -- 172.21.15.94:0/1397017290 shutdown_connections 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.646+0000 7f0922e00700 1 -- 172.21.15.94:0/1397017290 wait complete. 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 Processor -- start 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 -- start start 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c1046b0 0x7f091c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c105070 0x7f091c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c10a6b0 0x7f091c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:06.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f091c114a40 con 0x7f091c10a6b0 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1148c0 con 0x7f091c1046b0 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.647+0000 7f0922e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f091c114bc0 con 0x7f091c105070 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c1046b0 0x7f091c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c1046b0 0x7f091c19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:52274/0 (socket says 172.21.15.94:52274) 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091be00700 1 -- 172.21.15.94:0/2171689067 learned_addr learned my addr 172.21.15.94:0/2171689067 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091b400700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c105070 0x7f091c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f0920e00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c10a6b0 0x7f091c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091b400700 1 -- 172.21.15.94:0/2171689067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c1046b0 msgr2=0x7f091c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091b400700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c1046b0 0x7f091c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091b400700 1 -- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c10a6b0 msgr2=0x7f091c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091b400700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c10a6b0 0x7f091c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091b400700 1 -- 172.21.15.94:0/2171689067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c19ff00 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091be00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c1046b0 0x7f091c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.648+0000 7f091b400700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c105070 0x7f091c19b430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09080098e0 tx=0x7f090800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.649+0000 7f0918c00700 1 -- 172.21.15.94:0/2171689067 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908027070 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.649+0000 7f0918c00700 1 -- 172.21.15.94:0/2171689067 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f090800be60 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.649+0000 7f0918c00700 1 -- 172.21.15.94:0/2171689067 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908022dd0 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.649+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c1a0190 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.649+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c1a7a70 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.650+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091c069180 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.653+0000 7f0918c00700 1 -- 172.21.15.94:0/2171689067 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f090802a070 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.654+0000 7f0918c00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f09000a2ca0 0x7f09000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.654+0000 7f0918c00700 1 -- 172.21.15.94:0/2171689067 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09080d0220 con 0x7f091c105070 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.655+0000 7f091be00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f09000a2ca0 0x7f09000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.655+0000 7f091be00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f09000a2ca0 0x7f09000a5160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f090c009680 tx=0x7f090c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:06.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.656+0000 7f0918c00700 1 -- 172.21.15.94:0/2171689067 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0908013070 con 0x7f091c105070 2024-06-03T01:04:06.801 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:06 smithi094 ceph-mds[72981]: -- 172.21.15.94:0/2111755336 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56202260da40 con 0x5620225e0c00 2024-06-03T01:04:06.801 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:06 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d[72973]: starting mds.d at 2024-06-03T01:04:06.801 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:06 smithi094 bash[72940]: 95656a1bf6c1de201bc352574d965f7671b2e735712ec1ebcd661aa1da6317d6 2024-06-03T01:04:06.801 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:04:06 smithi094 systemd[1]: Started Ceph mds.d for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:06.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.937+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f091c02dca0 con 0x7f091c105070 2024-06-03T01:04:06.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.938+0000 7f0918c00700 1 -- 172.21.15.94:0/2171689067 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f090809d6a0 con 0x7f091c105070 2024-06-03T01:04:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:04:06.939 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-03T01:04:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f09000a2ca0 msgr2=0x7f09000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f09000a2ca0 0x7f09000a5160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f090c009680 tx=0x7f090c009260 comp rx=0 tx=0).stop 2024-06-03T01:04:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c105070 msgr2=0x7f091c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:06.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c105070 0x7f091c19b430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09080098e0 tx=0x7f090800b8a0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 shutdown_connections 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f091c1046b0 0x7f091c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f09000a2ca0 0x7f09000a5160 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f091c10a6b0 0x7f091c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 --2- 172.21.15.94:0/2171689067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f091c105070 0x7f091c19b430 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.941+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 >> 172.21.15.94:0/2171689067 conn(0x7f091c1001a0 msgr2=0x7f091c1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.942+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 shutdown_connections 2024-06-03T01:04:06.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:06.942+0000 7f0922e00700 1 -- 172.21.15.94:0/2171689067 wait complete. 2024-06-03T01:04:06.943 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 4 2024-06-03T01:04:07.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:07.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:07.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:07.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:07.389 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:07 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2171689067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:07.567 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:07.567 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.567 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.568 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.568 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:07.568 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:07.568 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:07.568 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:07 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2171689067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:07 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2171689067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:08 smithi094 ceph-mon[32119]: Deploying daemon mds.e on smithi107 2024-06-03T01:04:08.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:08 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.94:6836/3219738188,v1:172.21.15.94:6837/3219738188] up:boot 2024-06-03T01:04:08.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:08 smithi094 ceph-mon[32119]: fsmap 4 up:standby 2024-06-03T01:04:08.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:08 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T01:04:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:08 smithi179 ceph-mon[37041]: Deploying daemon mds.e on smithi107 2024-06-03T01:04:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:08 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.94:6836/3219738188,v1:172.21.15.94:6837/3219738188] up:boot 2024-06-03T01:04:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:08 smithi179 ceph-mon[37041]: fsmap 4 up:standby 2024-06-03T01:04:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:08 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T01:04:08.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:08 smithi107 ceph-mon[36622]: Deploying daemon mds.e on smithi107 2024-06-03T01:04:08.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:08 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.94:6836/3219738188,v1:172.21.15.94:6837/3219738188] up:boot 2024-06-03T01:04:08.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:08 smithi107 ceph-mon[36622]: fsmap 4 up:standby 2024-06-03T01:04:08.692 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:08 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T01:04:09.033 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:08 smithi107 systemd[1]: Starting Ceph mds.e for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:09.354 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:09 smithi107 ceph-mon[36622]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:09.355 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 podman[66382]: 2024-06-03 01:04:09.077404768 +0000 UTC m=+0.106979319 container create b23cdaba92a445658ea3e910019f9fc89af5e94a6ec94507cad41aa6c5f3e99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5) 2024-06-03T01:04:09.355 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 podman[66382]: 2024-06-03 01:04:08.987212172 +0000 UTC m=+0.016786669 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:09.355 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 podman[66382]: 2024-06-03 01:04:09.355269474 +0000 UTC m=+0.384843953 container init b23cdaba92a445658ea3e910019f9fc89af5e94a6ec94507cad41aa6c5f3e99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:04:09.542 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:09 smithi094 ceph-mon[32119]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:09 smithi179 ceph-mon[37041]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:09.783 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 podman[66382]: 2024-06-03 01:04:09.358031088 +0000 UTC m=+0.387605586 container start b23cdaba92a445658ea3e910019f9fc89af5e94a6ec94507cad41aa6c5f3e99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:04:09.783 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 ceph-mds[66401]: -- 172.21.15.107:0/1394516713 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565024b45a40 con 0x565024b18800 2024-06-03T01:04:09.783 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e[66397]: starting mds.e at 2024-06-03T01:04:09.783 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 bash[66382]: b23cdaba92a445658ea3e910019f9fc89af5e94a6ec94507cad41aa6c5f3e99a 2024-06-03T01:04:09.783 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:04:09 smithi107 systemd[1]: Started Ceph mds.e for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:09.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:04:10.269 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: Deploying daemon mds.f on smithi179 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:10.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6834/2528321857,v1:172.21.15.107:6835/2528321857] up:boot 2024-06-03T01:04:10.747 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: fsmap 5 up:standby 2024-06-03T01:04:10.747 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:10 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: Deploying daemon mds.f on smithi179 2024-06-03T01:04:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:10.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6834/2528321857,v1:172.21.15.107:6835/2528321857] up:boot 2024-06-03T01:04:10.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: fsmap 5 up:standby 2024-06-03T01:04:10.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:10 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:04:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: Deploying daemon mds.f on smithi179 2024-06-03T01:04:10.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:10.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6834/2528321857,v1:172.21.15.107:6835/2528321857] up:boot 2024-06-03T01:04:10.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: fsmap 5 up:standby 2024-06-03T01:04:10.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:10 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:04:12.037 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:11 smithi179 systemd[1]: Starting Ceph mds.f for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:12.315 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 podman[67315]: 2024-06-03 01:04:11.940919953 +0000 UTC m=+0.017344171 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:12.315 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 podman[67315]: 2024-06-03 01:04:12.06292505 +0000 UTC m=+0.139349261 container create 11b6c45346f147de94057d488e52d3b97415d1acac0e2c2d856040591fa8a41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-03T01:04:12.315 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 podman[67315]: 2024-06-03 01:04:12.315903382 +0000 UTC m=+0.392327591 container init 11b6c45346f147de94057d488e52d3b97415d1acac0e2c2d856040591fa8a41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-03T01:04:12.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 -- 172.21.15.94:0/4011941127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8101440 msgr2=0x7f58e810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:12.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 --2- 172.21.15.94:0/4011941127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8101440 0x7f58e810f330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b0a0 tx=0x7f58e000b3b0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 -- 172.21.15.94:0/4011941127 shutdown_connections 2024-06-03T01:04:12.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 --2- 172.21.15.94:0/4011941127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8101440 0x7f58e810f330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 --2- 172.21.15.94:0/4011941127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58e8108510 0x7f58e81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 --2- 172.21.15.94:0/4011941127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8100a80 0x7f58e8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 -- 172.21.15.94:0/4011941127 >> 172.21.15.94:0/4011941127 conn(0x7f58e8077a00 msgr2=0x7f58e80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 -- 172.21.15.94:0/4011941127 shutdown_connections 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.491+0000 7f58ef200700 1 -- 172.21.15.94:0/4011941127 wait complete. 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.492+0000 7f58ef200700 1 Processor -- start 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ef200700 1 -- start start 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ef200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8100a80 0x7f58e819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ef200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58e8101440 0x7f58e819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ef200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8108510 0x7f58e819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ef200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58e81127e0 con 0x7f58e8101440 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ef200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8112660 con 0x7f58e8100a80 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ef200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8112960 con 0x7f58e8108510 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58e7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8100a80 0x7f58e819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:12.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58e7400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58e8101440 0x7f58e819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ed200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8108510 0x7f58e819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58e7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8100a80 0x7f58e819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:52302/0 (socket says 172.21.15.94:52302) 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58e7e00700 1 -- 172.21.15.94:0/3875466168 learned_addr learned my addr 172.21.15.94:0/3875466168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.493+0000 7f58ed200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8108510 0x7f58e819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:48764/0 (socket says 172.21.15.94:48764) 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58ed200700 1 -- 172.21.15.94:0/3875466168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8100a80 msgr2=0x7f58e819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58ed200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8100a80 0x7f58e819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58ed200700 1 -- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58e8101440 msgr2=0x7f58e819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58ed200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58e8101440 0x7f58e819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58ed200700 1 -- 172.21.15.94:0/3875466168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e819fdf0 con 0x7f58e8108510 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58ed200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8108510 0x7f58e819f6b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b070 tx=0x7f58e0004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58e4c00700 1 -- 172.21.15.94:0/3875466168 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0021070 con 0x7f58e8108510 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.494+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e81a0080 con 0x7f58e8108510 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.495+0000 7f58e4c00700 1 -- 172.21.15.94:0/3875466168 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e0007b80 con 0x7f58e8108510 2024-06-03T01:04:12.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.495+0000 7f58e4c00700 1 -- 172.21.15.94:0/3875466168 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0005040 con 0x7f58e8108510 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.495+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e81a7960 con 0x7f58e8108510 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.496+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e8069180 con 0x7f58e8108510 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.497+0000 7f58e4c00700 1 -- 172.21.15.94:0/3875466168 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f58e0024070 con 0x7f58e8108510 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.499+0000 7f58e4c00700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f58cc0a2c20 0x7f58cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.499+0000 7f58e4c00700 1 -- 172.21.15.94:0/3875466168 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f58e000f020 con 0x7f58e8108510 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.499+0000 7f58e7e00700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f58cc0a2c20 0x7f58cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.500+0000 7f58e7e00700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f58cc0a2c20 0x7f58cc0a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009680 tx=0x7f58d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:12.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.503+0000 7f58e4c00700 1 -- 172.21.15.94:0/3875466168 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f58e002b020 con 0x7f58e8108510 2024-06-03T01:04:12.605 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 podman[67315]: 2024-06-03 01:04:12.318732953 +0000 UTC m=+0.395157180 container start 11b6c45346f147de94057d488e52d3b97415d1acac0e2c2d856040591fa8a41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:04:12.605 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 ceph-mds[67334]: -- 172.21.15.179:0/685687354 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abfdfa7a40 con 0x55abfdf7b000 2024-06-03T01:04:12.605 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f[67330]: starting mds.f at 2024-06-03T01:04:12.605 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 bash[67315]: 11b6c45346f147de94057d488e52d3b97415d1acac0e2c2d856040591fa8a41e 2024-06-03T01:04:12.605 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:04:12 smithi179 systemd[1]: Started Ceph mds.f for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:12.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.771+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58e81a7b30 con 0x7f58e8108510 2024-06-03T01:04:12.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.772+0000 7f58e4c00700 1 -- 172.21.15.94:0/3875466168 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f58e009d3c0 con 0x7f58e8108510 2024-06-03T01:04:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:04:12.773 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-03T01:04:12.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.774+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f58cc0a2c20 msgr2=0x7f58cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:12.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.774+0000 7f58ef200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f58cc0a2c20 0x7f58cc0a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58d8009680 tx=0x7f58d8009260 comp rx=0 tx=0).stop 2024-06-03T01:04:12.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.774+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8108510 msgr2=0x7f58e819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:12.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.774+0000 7f58ef200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8108510 0x7f58e819f6b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b070 tx=0x7f58e0004c80 comp rx=0 tx=0).stop 2024-06-03T01:04:12.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 shutdown_connections 2024-06-03T01:04:12.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8100a80 0x7f58e819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f58cc0a2c20 0x7f58cc0a50e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f58e8108510 0x7f58e819f6b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 --2- 172.21.15.94:0/3875466168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58e8101440 0x7f58e819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:12.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 >> 172.21.15.94:0/3875466168 conn(0x7f58e8077a00 msgr2=0x7f58e80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:12.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 shutdown_connections 2024-06-03T01:04:12.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:12.775+0000 7f58ef200700 1 -- 172.21.15.94:0/3875466168 wait complete. 2024-06-03T01:04:12.777 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 6 2024-06-03T01:04:13.472 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:13.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:13.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:13.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:13.474 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:13 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3875466168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:13 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3875466168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:13 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3875466168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:14.530 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:14 smithi179 ceph-mon[37041]: Deploying daemon mds.c on smithi179 2024-06-03T01:04:14.530 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] up:boot 2024-06-03T01:04:14.530 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:14 smithi179 ceph-mon[37041]: fsmap 6 up:standby 2024-06-03T01:04:14.530 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:14 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:04:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:14 smithi094 ceph-mon[32119]: Deploying daemon mds.c on smithi179 2024-06-03T01:04:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] up:boot 2024-06-03T01:04:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:14 smithi094 ceph-mon[32119]: fsmap 6 up:standby 2024-06-03T01:04:14.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:14 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:04:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:14 smithi107 ceph-mon[36622]: Deploying daemon mds.c on smithi179 2024-06-03T01:04:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] up:boot 2024-06-03T01:04:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:14 smithi107 ceph-mon[36622]: fsmap 6 up:standby 2024-06-03T01:04:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:14 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:04:14.855 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:14 smithi179 systemd[1]: Starting Ceph mds.c for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:15.250 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:14 smithi179 podman[67619]: 2024-06-03 01:04:14.828085612 +0000 UTC m=+0.016880954 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:15.250 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:14 smithi179 podman[67619]: 2024-06-03 01:04:14.947913161 +0000 UTC m=+0.136708477 container create 18ef3204d40437389fa34fa41e264f0ca277a2d00d71dce6e294f29db3ba97f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:04:15.250 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:15 smithi179 podman[67619]: 2024-06-03 01:04:15.251029543 +0000 UTC m=+0.439824859 container init 18ef3204d40437389fa34fa41e264f0ca277a2d00d71dce6e294f29db3ba97f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-03T01:04:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:15 smithi094 ceph-mon[32119]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:15 smithi179 ceph-mon[37041]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:15.605 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:15 smithi179 podman[67619]: 2024-06-03 01:04:15.253709031 +0000 UTC m=+0.442504363 container start 18ef3204d40437389fa34fa41e264f0ca277a2d00d71dce6e294f29db3ba97f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-03T01:04:15.605 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:15 smithi179 ceph-mds[67639]: -- 172.21.15.179:0/1301667068 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55556108fa40 con 0x555561062c00 2024-06-03T01:04:15.605 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c[67635]: starting mds.c at 2024-06-03T01:04:15.605 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:15 smithi179 bash[67619]: 18ef3204d40437389fa34fa41e264f0ca277a2d00d71dce6e294f29db3ba97f0 2024-06-03T01:04:15.605 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:04:15 smithi179 systemd[1]: Started Ceph mds.c for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:15.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:04:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:15 smithi107 ceph-mon[36622]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:16.101 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:16.346 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] up:boot 2024-06-03T01:04:16.346 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: fsmap 7 up:standby 2024-06-03T01:04:16.346 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: Deploying daemon mds.h on smithi107 2024-06-03T01:04:16.347 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:16 smithi107 ceph-mon[36622]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] up:boot 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: fsmap 7 up:standby 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:16.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:16.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: Deploying daemon mds.h on smithi107 2024-06-03T01:04:16.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:16 smithi094 ceph-mon[32119]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] up:boot 2024-06-03T01:04:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: fsmap 7 up:standby 2024-06-03T01:04:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:04:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:16.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:16.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:16.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: Deploying daemon mds.h on smithi107 2024-06-03T01:04:16.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:16 smithi179 ceph-mon[37041]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:17.783 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:17 smithi107 systemd[1]: Starting Ceph mds.h for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:18.084 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:17 smithi107 podman[66683]: 2024-06-03 01:04:17.830916114 +0000 UTC m=+0.100365421 container create a5069b67326673ece68b0cdf30f815cb1f2248140166ad7befd4bc9c39cdcad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:04:18.084 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:17 smithi107 podman[66683]: 2024-06-03 01:04:17.747696344 +0000 UTC m=+0.017145657 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:18.084 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:18 smithi107 podman[66683]: 2024-06-03 01:04:18.058550806 +0000 UTC m=+0.328000113 container init a5069b67326673ece68b0cdf30f815cb1f2248140166ad7befd4bc9c39cdcad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-03T01:04:18.084 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:18 smithi107 podman[66683]: 2024-06-03 01:04:18.061184523 +0000 UTC m=+0.330633845 container start a5069b67326673ece68b0cdf30f815cb1f2248140166ad7befd4bc9c39cdcad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-03T01:04:18.084 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:18 smithi107 ceph-mds[66703]: -- 172.21.15.107:0/3501801513 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a956d7a40 con 0x563a956ab000 2024-06-03T01:04:18.533 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h[66699]: starting mds.h at 2024-06-03T01:04:18.533 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:18 smithi107 bash[66683]: a5069b67326673ece68b0cdf30f815cb1f2248140166ad7befd4bc9c39cdcad1 2024-06-03T01:04:18.533 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:04:18 smithi107 systemd[1]: Started Ceph mds.h for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:18.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 -- 172.21.15.94:0/159722650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8113660 msgr2=0x7f93b8115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:18.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 --2- 172.21.15.94:0/159722650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8113660 0x7f93b8115a50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f93b000b0a0 tx=0x7f93b000b3b0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 -- 172.21.15.94:0/159722650 shutdown_connections 2024-06-03T01:04:18.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 --2- 172.21.15.94:0/159722650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8113660 0x7f93b8115a50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 --2- 172.21.15.94:0/159722650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b80746c0 0x7f93b8074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 --2- 172.21.15.94:0/159722650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 0x7f93b8113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 -- 172.21.15.94:0/159722650 >> 172.21.15.94:0/159722650 conn(0x7f93b80795f0 msgr2=0x7f93b8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:18.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 -- 172.21.15.94:0/159722650 shutdown_connections 2024-06-03T01:04:18.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.592+0000 7f93bcc00700 1 -- 172.21.15.94:0/159722650 wait complete. 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 Processor -- start 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 -- start start 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b80746c0 0x7f93b819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 0x7f93b819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b8113660 0x7f93b81a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8118df0 con 0x7f93b8113660 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8118c70 con 0x7f93b80746c0 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93bcc00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8118f70 con 0x7f93b8074fe0 2024-06-03T01:04:18.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.594+0000 7f93b6800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b8113660 0x7f93b81a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 0x7f93b819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b80746c0 0x7f93b819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b80746c0 0x7f93b819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:60884/0 (socket says 172.21.15.94:60884) 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 0x7f93b819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38186/0 (socket says 172.21.15.94:38186) 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5e00700 1 -- 172.21.15.94:0/3467427236 learned_addr learned my addr 172.21.15.94:0/3467427236 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 -- 172.21.15.94:0/3467427236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b80746c0 msgr2=0x7f93b819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b80746c0 0x7f93b819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 -- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b8113660 msgr2=0x7f93b81a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b8113660 0x7f93b81a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 -- 172.21.15.94:0/3467427236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b81a4270 con 0x7f93b8074fe0 2024-06-03T01:04:18.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93b5400700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 0x7f93b819f7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e9f0 tx=0x7f93a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:18.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.595+0000 7f93aea00700 1 -- 172.21.15.94:0/3467427236 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4004220 con 0x7f93b8074fe0 2024-06-03T01:04:18.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.596+0000 7f93aea00700 1 -- 172.21.15.94:0/3467427236 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a4004d10 con 0x7f93b8074fe0 2024-06-03T01:04:18.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.596+0000 7f93aea00700 1 -- 172.21.15.94:0/3467427236 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4005020 con 0x7f93b8074fe0 2024-06-03T01:04:18.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.596+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b81a4560 con 0x7f93b8074fe0 2024-06-03T01:04:18.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.596+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b81abde0 con 0x7f93b8074fe0 2024-06-03T01:04:18.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.597+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b8069180 con 0x7f93b8074fe0 2024-06-03T01:04:18.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.599+0000 7f93aea00700 1 -- 172.21.15.94:0/3467427236 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f93a4004750 con 0x7f93b8074fe0 2024-06-03T01:04:18.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.600+0000 7f93aea00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f939c0a2ca0 0x7f939c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:18.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.601+0000 7f93aea00700 1 -- 172.21.15.94:0/3467427236 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f93a40cea10 con 0x7f93b8074fe0 2024-06-03T01:04:18.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.601+0000 7f93b5e00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f939c0a2ca0 0x7f939c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:18.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.601+0000 7f93b5e00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f939c0a2ca0 0x7f939c0a5160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009d20 tx=0x7f93a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:18.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.605+0000 7f93aea00700 1 -- 172.21.15.94:0/3467427236 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f93a409bd70 con 0x7f93b8074fe0 2024-06-03T01:04:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.873+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93b81a05c0 con 0x7f93b8074fe0 2024-06-03T01:04:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.875+0000 7f93aea00700 1 -- 172.21.15.94:0/3467427236 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f93a409b670 con 0x7f93b8074fe0 2024-06-03T01:04:18.875 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:04:18.875 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-03T01:04:18.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f939c0a2ca0 msgr2=0x7f939c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:18.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f939c0a2ca0 0x7f939c0a5160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009d20 tx=0x7f93a8009260 comp rx=0 tx=0).stop 2024-06-03T01:04:18.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 msgr2=0x7f93b819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:18.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 0x7f93b819f7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e9f0 tx=0x7f93a400ed00 comp rx=0 tx=0).stop 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 shutdown_connections 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b80746c0 0x7f93b819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f939c0a2ca0 0x7f939c0a5160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93b8113660 0x7f93b81a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 --2- 172.21.15.94:0/3467427236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93b8074fe0 0x7f93b819f7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 >> 172.21.15.94:0/3467427236 conn(0x7f93b80795f0 msgr2=0x7f93b8112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 shutdown_connections 2024-06-03T01:04:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:18.878+0000 7f93bcc00700 1 -- 172.21.15.94:0/3467427236 wait complete. 2024-06-03T01:04:18.880 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] up:boot 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: fsmap 8 up:standby 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: Deploying daemon mds.i on smithi179 2024-06-03T01:04:19.155 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:19 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3467427236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] up:boot 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: fsmap 8 up:standby 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:19.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:19.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:19.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:19.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: Deploying daemon mds.i on smithi179 2024-06-03T01:04:19.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:19 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3467427236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] up:boot 2024-06-03T01:04:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: fsmap 8 up:standby 2024-06-03T01:04:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:04:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:04:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:19.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:19.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:19.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:19.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:19.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: Deploying daemon mds.i on smithi179 2024-06-03T01:04:19.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:19 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3467427236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:20.605 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:20 smithi179 systemd[1]: Starting Ceph mds.i for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:20.991 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:20 smithi179 podman[67937]: 2024-06-03 01:04:20.640255717 +0000 UTC m=+0.017085721 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:20.991 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:20 smithi179 podman[67937]: 2024-06-03 01:04:20.755423653 +0000 UTC m=+0.132253684 container create e0f7f4e23e85c9fb5c11e98394b64865c6fb94962261b8c6b5252f649691e417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:04:20.991 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:20 smithi179 podman[67937]: 2024-06-03 01:04:20.991626754 +0000 UTC m=+0.368456739 container init e0f7f4e23e85c9fb5c11e98394b64865c6fb94962261b8c6b5252f649691e417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:04:21.304 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:20 smithi179 podman[67937]: 2024-06-03 01:04:20.994346655 +0000 UTC m=+0.371176648 container start e0f7f4e23e85c9fb5c11e98394b64865c6fb94962261b8c6b5252f649691e417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True) 2024-06-03T01:04:21.305 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mds[67956]: -- 172.21.15.179:0/1591877813 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578f0abfa40 con 0x5578f0a93000 2024-06-03T01:04:21.305 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i[67952]: starting mds.i at 2024-06-03T01:04:21.305 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:21 smithi179 bash[67937]: e0f7f4e23e85c9fb5c11e98394b64865c6fb94962261b8c6b5252f649691e417 2024-06-03T01:04:21.305 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:04:21 smithi179 systemd[1]: Started Ceph mds.i for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:21.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:21 smithi094 ceph-mon[32119]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:21.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:21.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:21.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:21 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mon[37041]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:21 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:21.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:04:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:21 smithi107 ceph-mon[36622]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:21.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:21.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:21 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:22.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:22 smithi094 ceph-mon[32119]: Deploying daemon mds.g on smithi094 2024-06-03T01:04:22.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:22 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] up:boot 2024-06-03T01:04:22.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:22 smithi094 ceph-mon[32119]: fsmap 9 up:standby 2024-06-03T01:04:22.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:22 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:04:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:22 smithi179 ceph-mon[37041]: Deploying daemon mds.g on smithi094 2024-06-03T01:04:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:22 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] up:boot 2024-06-03T01:04:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:22 smithi179 ceph-mon[37041]: fsmap 9 up:standby 2024-06-03T01:04:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:22 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:04:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:22 smithi107 ceph-mon[36622]: Deploying daemon mds.g on smithi094 2024-06-03T01:04:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:22 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] up:boot 2024-06-03T01:04:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:22 smithi107 ceph-mon[36622]: fsmap 9 up:standby 2024-06-03T01:04:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:22 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:04:22.971 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:23.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:23 smithi094 ceph-mon[32119]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:23 smithi179 ceph-mon[37041]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:23.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:23 smithi107 ceph-mon[36622]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:24.850 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:24 smithi094 systemd[1]: Starting Ceph mds.g for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:25.585 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:25 smithi107 ceph-mon[36622]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:25.592 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:25 smithi094 ceph-mon[32119]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:25 smithi179 ceph-mon[37041]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:25.843 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:25 smithi094 podman[73717]: 2024-06-03 01:04:25.502771873 +0000 UTC m=+0.041496854 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:25.843 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:25 smithi094 podman[73717]: 2024-06-03 01:04:25.618221855 +0000 UTC m=+0.156946850 container create 3daa0df28bb214e4880d4c8d1d5cbd15f7322362c1acc34511b4e8c688ef38b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-03T01:04:26.182 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:26 smithi094 podman[73717]: 2024-06-03 01:04:26.065398344 +0000 UTC m=+0.604123402 container init 3daa0df28bb214e4880d4c8d1d5cbd15f7322362c1acc34511b4e8c688ef38b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-03T01:04:26.182 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:26 smithi094 podman[73717]: 2024-06-03 01:04:26.075198283 +0000 UTC m=+0.613923280 container start 3daa0df28bb214e4880d4c8d1d5cbd15f7322362c1acc34511b4e8c688ef38b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-03T01:04:26.182 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:26 smithi094 ceph-mds[73754]: -- 172.21.15.94:0/1933679470 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e46cf9da40 con 0x55e46cf71000 2024-06-03T01:04:26.182 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g[73749]: starting mds.g at 2024-06-03T01:04:26.182 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:26 smithi094 bash[73717]: 3daa0df28bb214e4880d4c8d1d5cbd15f7322362c1acc34511b4e8c688ef38b6 2024-06-03T01:04:26.447 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:04:26 smithi094 systemd[1]: Started Ceph mds.g for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 -- 172.21.15.94:0/1692414529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb010c9b0 msgr2=0x7fceb010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 --2- 172.21.15.94:0/1692414529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb010c9b0 0x7fceb010ee70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009990 tx=0x7fcea8009ca0 comp rx=0 tx=0).stop 2024-06-03T01:04:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 -- 172.21.15.94:0/1692414529 shutdown_connections 2024-06-03T01:04:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 --2- 172.21.15.94:0/1692414529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb010c9b0 0x7fceb010ee70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 --2- 172.21.15.94:0/1692414529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb0103580 0x7fceb0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 --2- 172.21.15.94:0/1692414529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb0103ea0 0x7fceb010c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:26.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 -- 172.21.15.94:0/1692414529 >> 172.21.15.94:0/1692414529 conn(0x7fceb00fcdb0 msgr2=0x7fceb00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:26.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.746+0000 7fceb6a00700 1 -- 172.21.15.94:0/1692414529 shutdown_connections 2024-06-03T01:04:26.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.747+0000 7fceb6a00700 1 -- 172.21.15.94:0/1692414529 wait complete. 2024-06-03T01:04:26.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.748+0000 7fceb6a00700 1 Processor -- start 2024-06-03T01:04:26.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.748+0000 7fceb6a00700 1 -- start start 2024-06-03T01:04:26.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb0103580 0x7fceb01a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb0103ea0 0x7fceb01a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb6a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb010c9b0 0x7fceb010b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb6a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0112750 con 0x7fceb0103580 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb6a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01125d0 con 0x7fceb0103ea0 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb6a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fceb01128d0 con 0x7fceb010c9b0 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb4c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb0103ea0 0x7fceb01a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb4c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb0103ea0 0x7fceb01a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:38142/0 (socket says 172.21.15.94:38142) 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb4c00700 1 -- 172.21.15.94:0/3959691799 learned_addr learned my addr 172.21.15.94:0/3959691799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:26.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb5600700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb0103580 0x7fceb01a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.749+0000 7fceb6000700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb010c9b0 0x7fceb010b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb4c00700 1 -- 172.21.15.94:0/3959691799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb010c9b0 msgr2=0x7fceb010b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb4c00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb010c9b0 0x7fceb010b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb4c00700 1 -- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb0103580 msgr2=0x7fceb01a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb4c00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb0103580 0x7fceb01a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb4c00700 1 -- 172.21.15.94:0/3959691799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb010bd80 con 0x7fceb0103ea0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb4c00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb0103ea0 0x7fceb01a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e9c0 tx=0x7fce9c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceae000700 1 -- 172.21.15.94:0/3959691799 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c004a30 con 0x7fceb0103ea0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceae000700 1 -- 172.21.15.94:0/3959691799 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce9c00cca0 con 0x7fceb0103ea0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceae000700 1 -- 172.21.15.94:0/3959691799 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c005610 con 0x7fceb0103ea0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb010c070 con 0x7fceb0103ea0 2024-06-03T01:04:26.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.750+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb01abd00 con 0x7fceb0103ea0 2024-06-03T01:04:26.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.751+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb0069180 con 0x7fceb0103ea0 2024-06-03T01:04:26.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.753+0000 7fceae000700 1 -- 172.21.15.94:0/3959691799 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fce9c00c7c0 con 0x7fceb0103ea0 2024-06-03T01:04:26.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.755+0000 7fceae000700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce980a2c20 0x7fce980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:26.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.756+0000 7fceae000700 1 -- 172.21.15.94:0/3959691799 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fce9c0cece0 con 0x7fceb0103ea0 2024-06-03T01:04:26.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.756+0000 7fceb5600700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce980a2c20 0x7fce980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:26.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.756+0000 7fceb5600700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce980a2c20 0x7fce980a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcea400f540 tx=0x7fcea400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:26.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:26.763+0000 7fceae000700 1 -- 172.21.15.94:0/3959691799 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fce9c09c040 con 0x7fceb0103ea0 2024-06-03T01:04:27.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.045+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceb01a0520 con 0x7fceb0103ea0 2024-06-03T01:04:27.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.046+0000 7fceae000700 1 -- 172.21.15.94:0/3959691799 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7fce9c09b940 con 0x7fceb0103ea0 2024-06-03T01:04:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:04:27.046 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-03T01:04:27.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.048+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce980a2c20 msgr2=0x7fce980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:27.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce980a2c20 0x7fce980a50e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcea400f540 tx=0x7fcea400b040 comp rx=0 tx=0).stop 2024-06-03T01:04:27.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb0103ea0 msgr2=0x7fceb01a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:27.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb0103ea0 0x7fceb01a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00e9c0 tx=0x7fce9c00ecd0 comp rx=0 tx=0).stop 2024-06-03T01:04:27.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 shutdown_connections 2024-06-03T01:04:27.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce980a2c20 0x7fce980a50e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:27.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fceb010c9b0 0x7fceb010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:27.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fceb0103580 0x7fceb01a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:27.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 --2- 172.21.15.94:0/3959691799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fceb0103ea0 0x7fceb01a53e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:27.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 >> 172.21.15.94:0/3959691799 conn(0x7fceb00fcdb0 msgr2=0x7fceb010ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:27.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.049+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 shutdown_connections 2024-06-03T01:04:27.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:27.050+0000 7fceb6a00700 1 -- 172.21.15.94:0/3959691799 wait complete. 2024-06-03T01:04:27.051 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 11 2024-06-03T01:04:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.94:6838/848723917,v1:172.21.15.94:6839/848723917] up:boot 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: fsmap 10 up:standby 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:27 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3959691799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.94:6838/848723917,v1:172.21.15.94:6839/848723917] up:boot 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: fsmap 10 up:standby 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:04:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:27.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:27.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:27.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:27 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3959691799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.94:6838/848723917,v1:172.21.15.94:6839/848723917] up:boot 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: fsmap 10 up:standby 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:27.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:27 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3959691799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:28.526 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:28 smithi094 ceph-mon[32119]: Deploying daemon mds.a on smithi094 2024-06-03T01:04:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:28 smithi179 ceph-mon[37041]: Deploying daemon mds.a on smithi094 2024-06-03T01:04:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:28 smithi107 ceph-mon[36622]: Deploying daemon mds.a on smithi094 2024-06-03T01:04:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:29 smithi094 ceph-mon[32119]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:29 smithi179 ceph-mon[37041]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:29 smithi107 ceph-mon[36622]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:29.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:04:30.349 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 systemd[1]: Starting Ceph mds.a for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:30.796 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 podman[74119]: 2024-06-03 01:04:30.382289645 +0000 UTC m=+0.039877011 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:30.796 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 podman[74119]: 2024-06-03 01:04:30.482827032 +0000 UTC m=+0.140414435 container create 9f54f277a773ee40ee34e26ab5aac76a06d5491abae76c9f220512613dd9b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528) 2024-06-03T01:04:30.796 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 podman[74119]: 2024-06-03 01:04:30.795619492 +0000 UTC m=+0.453206901 container init 9f54f277a773ee40ee34e26ab5aac76a06d5491abae76c9f220512613dd9b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-03T01:04:30.910 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:31.060 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 podman[74119]: 2024-06-03 01:04:30.805861461 +0000 UTC m=+0.463448856 container start 9f54f277a773ee40ee34e26ab5aac76a06d5491abae76c9f220512613dd9b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:04:31.060 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 ceph-mds[74158]: -- 172.21.15.94:0/513308899 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651e277da40 con 0x5651e2751000 2024-06-03T01:04:31.060 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a[74154]: starting mds.a at 2024-06-03T01:04:31.060 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 bash[74119]: 9f54f277a773ee40ee34e26ab5aac76a06d5491abae76c9f220512613dd9b2c1 2024-06-03T01:04:31.060 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:04:30 smithi094 systemd[1]: Started Ceph mds.a for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:31.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:31 smithi094 ceph-mon[32119]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:31.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:31.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:31.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:31 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:31 smithi179 ceph-mon[37041]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:31.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:31 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:31.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:31 smithi107 ceph-mon[36622]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:31.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:31.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-03T01:04:31.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-03T01:04:31.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:31 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:32 smithi094 ceph-mon[32119]: Deploying daemon mds.b on smithi107 2024-06-03T01:04:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:32 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.94:6840/3437199777,v1:172.21.15.94:6841/3437199777] up:boot 2024-06-03T01:04:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:32 smithi094 ceph-mon[32119]: fsmap 11 up:standby 2024-06-03T01:04:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:32 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:04:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:32 smithi179 ceph-mon[37041]: Deploying daemon mds.b on smithi107 2024-06-03T01:04:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:32 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.94:6840/3437199777,v1:172.21.15.94:6841/3437199777] up:boot 2024-06-03T01:04:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:32 smithi179 ceph-mon[37041]: fsmap 11 up:standby 2024-06-03T01:04:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:32 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:04:32.664 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:32 smithi107 ceph-mon[36622]: Deploying daemon mds.b on smithi107 2024-06-03T01:04:32.665 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:32 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.94:6840/3437199777,v1:172.21.15.94:6841/3437199777] up:boot 2024-06-03T01:04:32.665 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:32 smithi107 ceph-mon[36622]: fsmap 11 up:standby 2024-06-03T01:04:32.665 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:32 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:04:33.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.256+0000 7f185da00700 1 -- 172.21.15.94:0/896658472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 msgr2=0x7f1858107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:33.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.256+0000 7f185da00700 1 --2- 172.21.15.94:0/896658472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 0x7f1858107110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18400099c0 tx=0x7f1840009cd0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.256+0000 7f185da00700 1 -- 172.21.15.94:0/896658472 shutdown_connections 2024-06-03T01:04:33.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.256+0000 7f185da00700 1 --2- 172.21.15.94:0/896658472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 0x7f1858102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.256+0000 7f185da00700 1 --2- 172.21.15.94:0/896658472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1858107840 0x7f185810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.256+0000 7f185da00700 1 --2- 172.21.15.94:0/896658472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 0x7f1858107110 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.256+0000 7f185da00700 1 -- 172.21.15.94:0/896658472 >> 172.21.15.94:0/896658472 conn(0x7f18580795f0 msgr2=0x7f1858079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:33.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.257+0000 7f185da00700 1 -- 172.21.15.94:0/896658472 shutdown_connections 2024-06-03T01:04:33.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.257+0000 7f185da00700 1 -- 172.21.15.94:0/896658472 wait complete. 2024-06-03T01:04:33.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.258+0000 7f185da00700 1 Processor -- start 2024-06-03T01:04:33.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.258+0000 7f185da00700 1 -- start start 2024-06-03T01:04:33.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f185da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 0x7f1858074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:33.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f185da00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 0x7f1858074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f185da00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1858107840 0x7f1858075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f185da00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1858112770 con 0x7f18581030f0 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f185da00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18581125f0 con 0x7f1858102740 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f185da00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f18581128f0 con 0x7f1858107840 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f1856a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 0x7f1858074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f184e000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 0x7f1858074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f1856a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 0x7f1858074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:38172/0 (socket says 172.21.15.94:38172) 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f1856a00700 1 -- 172.21.15.94:0/216539896 learned_addr learned my addr 172.21.15.94:0/216539896 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:33.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.259+0000 7f1857400700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1858107840 0x7f1858075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:33.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f1856a00700 1 -- 172.21.15.94:0/216539896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1858107840 msgr2=0x7f1858075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f1856a00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1858107840 0x7f1858075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f1856a00700 1 -- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 msgr2=0x7f1858074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f1856a00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 0x7f1858074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f1856a00700 1 -- 172.21.15.94:0/216539896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18581a7110 con 0x7f1858102740 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f1856a00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 0x7f1858074790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f184800ef10 tx=0x7f184800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f184fe00700 1 -- 172.21.15.94:0/216539896 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800c9c0 con 0x7f1858102740 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f184fe00700 1 -- 172.21.15.94:0/216539896 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1848019070 con 0x7f1858102740 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18581a73a0 con 0x7f1858102740 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.260+0000 7f184fe00700 1 -- 172.21.15.94:0/216539896 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848007e60 con 0x7f1858102740 2024-06-03T01:04:33.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.261+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a7900 con 0x7f1858102740 2024-06-03T01:04:33.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.262+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1858069180 con 0x7f1858102740 2024-06-03T01:04:33.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.263+0000 7f184fe00700 1 -- 172.21.15.94:0/216539896 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1848012050 con 0x7f1858102740 2024-06-03T01:04:33.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.265+0000 7f184fe00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f183c0a2ca0 0x7f183c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:33.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.265+0000 7f184fe00700 1 -- 172.21.15.94:0/216539896 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f18480ceb50 con 0x7f1858102740 2024-06-03T01:04:33.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.265+0000 7f184e000700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f183c0a2ca0 0x7f183c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:33.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.266+0000 7f184e000700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f183c0a2ca0 0x7f183c0a5160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1858076220 tx=0x7f1840005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:33.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.269+0000 7f184fe00700 1 -- 172.21.15.94:0/216539896 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1848010040 con 0x7f1858102740 2024-06-03T01:04:33.283 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 systemd[1]: Starting Ceph mds.b for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:04:33.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.540+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f185802de60 con 0x7f1858102740 2024-06-03T01:04:33.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.541+0000 7f184fe00700 1 -- 172.21.15.94:0/216539896 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f184809beb0 con 0x7f1858102740 2024-06-03T01:04:33.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:04:33.542 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-03T01:04:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.544+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f183c0a2ca0 msgr2=0x7f183c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.544+0000 7f185da00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f183c0a2ca0 0x7f183c0a5160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1858076220 tx=0x7f1840005fb0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.544+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 msgr2=0x7f1858074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.544+0000 7f185da00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 0x7f1858074790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f184800ef10 tx=0x7f184800c3a0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 shutdown_connections 2024-06-03T01:04:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1858102740 0x7f1858074790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f183c0a2ca0 0x7f183c0a5160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1858107840 0x7f1858075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 --2- 172.21.15.94:0/216539896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581030f0 0x7f1858074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 >> 172.21.15.94:0/216539896 conn(0x7f18580795f0 msgr2=0x7f1858100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 shutdown_connections 2024-06-03T01:04:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:33.545+0000 7f185da00700 1 -- 172.21.15.94:0/216539896 wait complete. 2024-06-03T01:04:33.546 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-06-03T01:04:33.594 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:33 smithi107 ceph-mon[36622]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:33.594 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 podman[66995]: 2024-06-03 01:04:33.250194283 +0000 UTC m=+0.016765635 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:04:33.594 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 podman[66995]: 2024-06-03 01:04:33.359202306 +0000 UTC m=+0.125773652 container create ab857863a9e0ae32331fe86e58ab955f464b11ef4d5b8984cd03e16f1fdb171c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-03T01:04:33.595 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 podman[66995]: 2024-06-03 01:04:33.57003174 +0000 UTC m=+0.336603087 container init ab857863a9e0ae32331fe86e58ab955f464b11ef4d5b8984cd03e16f1fdb171c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-03T01:04:33.595 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 podman[66995]: 2024-06-03 01:04:33.572511148 +0000 UTC m=+0.339082511 container start ab857863a9e0ae32331fe86e58ab955f464b11ef4d5b8984cd03e16f1fdb171c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-03T01:04:33.595 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 ceph-mds[67014]: -- 172.21.15.107:0/2938111389 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5e30eda40 con 0x55e5e30c1000 2024-06-03T01:04:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:33 smithi094 ceph-mon[32119]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:33 smithi179 ceph-mon[37041]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:34.033 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: starting mds.b at 2024-06-03T01:04:34.033 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 bash[66995]: ab857863a9e0ae32331fe86e58ab955f464b11ef4d5b8984cd03e16f1fdb171c 2024-06-03T01:04:34.033 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:04:33 smithi107 systemd[1]: Started Ceph mds.b for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:04:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/216539896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/216539896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/216539896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:04:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:34.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:35 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] up:boot 2024-06-03T01:04:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:35 smithi107 ceph-mon[36622]: fsmap 12 up:standby 2024-06-03T01:04:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:04:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:35 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] up:boot 2024-06-03T01:04:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:35 smithi094 ceph-mon[32119]: fsmap 12 up:standby 2024-06-03T01:04:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:04:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:35 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] up:boot 2024-06-03T01:04:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:35 smithi179 ceph-mon[37041]: fsmap 12 up:standby 2024-06-03T01:04:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:04:36.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:04:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:36 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:36 smithi094 ceph-mon[32119]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:36.950 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:37.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:36 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:36 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:36 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:36 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:36 smithi107 ceph-mon[36622]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:36 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:36 smithi179 ceph-mon[37041]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:38.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:04:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:37 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:04:38.194 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.194 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.194 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:38.194 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:04:38.194 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.194 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:37 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:04:38.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.195 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:38.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:04:38.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:38.196 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:37 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:04:39.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1612374723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd406a6b0 msgr2=0x7f9cd406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:39.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1612374723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd406a6b0 0x7f9cd406ab10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009990 tx=0x7f9cc0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:39 smithi094 ceph-mon[32119]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:39.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.300 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:39 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1612374723 shutdown_connections 2024-06-03T01:04:39.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1612374723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd4104e90 0x7f9cd4105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1612374723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cd4107380 0x7f9cd406a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1612374723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd406a6b0 0x7f9cd406ab10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1612374723 >> 172.21.15.94:0/1612374723 conn(0x7f9cd40782c0 msgr2=0x7f9cd40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:39.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1612374723 shutdown_connections 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.297+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1612374723 wait complete. 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.299+0000 7f9cd8c00700 1 Processor -- start 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.299+0000 7f9cd8c00700 1 -- start start 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.299+0000 7f9cd8c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd406a6b0 0x7f9cd419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.299+0000 7f9cd8c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cd4104e90 0x7f9cd419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd8c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4107380 0x7f9cd41a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd8c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4118d70 con 0x7f9cd4107380 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd8c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4118bf0 con 0x7f9cd406a6b0 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd8c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4118ef0 con 0x7f9cd4104e90 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd2800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4107380 0x7f9cd41a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd406a6b0 0x7f9cd419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:39.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd406a6b0 0x7f9cd419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47536/0 (socket says 172.21.15.94:47536) 2024-06-03T01:04:39.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 -- 172.21.15.94:0/1217681679 learned_addr learned my addr 172.21.15.94:0/1217681679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:39.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd2800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4107380 0x7f9cd41a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59378/0 (socket says 172.21.15.94:59378) 2024-06-03T01:04:39.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1400700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cd4104e90 0x7f9cd419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 -- 172.21.15.94:0/1217681679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cd4104e90 msgr2=0x7f9cd419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cd4104e90 0x7f9cd419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 -- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4107380 msgr2=0x7f9cd41a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4107380 0x7f9cd41a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.300+0000 7f9cd1e00700 1 -- 172.21.15.94:0/1217681679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd41a4270 con 0x7f9cd406a6b0 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.301+0000 7f9cd1e00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd406a6b0 0x7f9cd419f260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400e9a0 tx=0x7f9cc400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.301+0000 7f9ccaa00700 1 -- 172.21.15.94:0/1217681679 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400c9c0 con 0x7f9cd406a6b0 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.301+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd41a4560 con 0x7f9cd406a6b0 2024-06-03T01:04:39.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.301+0000 7f9ccaa00700 1 -- 172.21.15.94:0/1217681679 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc400cb60 con 0x7f9cd406a6b0 2024-06-03T01:04:39.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.301+0000 7f9ccaa00700 1 -- 172.21.15.94:0/1217681679 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4018680 con 0x7f9cd406a6b0 2024-06-03T01:04:39.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.301+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41abde0 con 0x7f9cd406a6b0 2024-06-03T01:04:39.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.303+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd4069180 con 0x7f9cd406a6b0 2024-06-03T01:04:39.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.304+0000 7f9ccaa00700 1 -- 172.21.15.94:0/1217681679 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9cc400cd10 con 0x7f9cd406a6b0 2024-06-03T01:04:39.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.305+0000 7f9ccaa00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9cb80a2c20 0x7f9cb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:39.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.305+0000 7f9ccaa00700 1 -- 172.21.15.94:0/1217681679 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9cc40ced20 con 0x7f9cd406a6b0 2024-06-03T01:04:39.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.305+0000 7f9cd1400700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9cb80a2c20 0x7f9cb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:39.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.306+0000 7f9cd1400700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9cb80a2c20 0x7f9cb80a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9cd406adf0 tx=0x7f9cc000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:39.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.309+0000 7f9ccaa00700 1 -- 172.21.15.94:0/1217681679 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9cc409c080 con 0x7f9cd406a6b0 2024-06-03T01:04:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:39 smithi179 ceph-mon[37041]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:39 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:39 smithi107 ceph-mon[36622]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:39 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:39.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.578+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cd41ac010 con 0x7f9cd406a6b0 2024-06-03T01:04:39.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.579+0000 7f9ccaa00700 1 -- 172.21.15.94:0/1217681679 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f9cc409b980 con 0x7f9cd406a6b0 2024-06-03T01:04:39.580 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:04:39.580 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.581+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9cb80a2c20 msgr2=0x7f9cb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.581+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9cb80a2c20 0x7f9cb80a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9cd406adf0 tx=0x7f9cc000b260 comp rx=0 tx=0).stop 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.581+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd406a6b0 msgr2=0x7f9cd419f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.581+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd406a6b0 0x7f9cd419f260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400e9a0 tx=0x7f9cc400ecb0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 shutdown_connections 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd406a6b0 0x7f9cd419f260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9cb80a2c20 0x7f9cb80a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cd4107380 0x7f9cd41a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 --2- 172.21.15.94:0/1217681679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cd4104e90 0x7f9cd419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 >> 172.21.15.94:0/1217681679 conn(0x7f9cd40782c0 msgr2=0x7f9cd4103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 shutdown_connections 2024-06-03T01:04:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:39.582+0000 7f9cd8c00700 1 -- 172.21.15.94:0/1217681679 wait complete. 2024-06-03T01:04:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 13 2024-06-03T01:04:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:40 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1217681679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:40 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1217681679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:40 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1217681679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:04:41.012 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-03T01:04:41.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:41 smithi094 ceph-mon[32119]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:41.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:41 smithi179 ceph-mon[37041]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:41.461 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:41.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:41 smithi107 ceph-mon[36622]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:43.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:43 smithi094 ceph-mon[32119]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:43.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:43.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:04:43.391 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:43 smithi179 ceph-mon[37041]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:04:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 -- 172.21.15.94:0/3592472344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34104680 msgr2=0x7fcd34104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3592472344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34104680 0x7fcd34104b00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009990 tx=0x7fcd20009ca0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 -- 172.21.15.94:0/3592472344 shutdown_connections 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3592472344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34105040 0x7fcd34111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3592472344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd3410a680 0x7fcd3410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3592472344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34104680 0x7fcd34104b00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 -- 172.21.15.94:0/3592472344 >> 172.21.15.94:0/3592472344 conn(0x7fcd34100150 msgr2=0x7fcd34102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 -- 172.21.15.94:0/3592472344 shutdown_connections 2024-06-03T01:04:43.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.744+0000 7fcd39e00700 1 -- 172.21.15.94:0/3592472344 wait complete. 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.745+0000 7fcd39e00700 1 Processor -- start 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.745+0000 7fcd39e00700 1 -- start start 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd39e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34104680 0x7fcd3419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd39e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34105040 0x7fcd3419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd39e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3410a680 0x7fcd3419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd39e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341149b0 con 0x7fcd3410a680 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd39e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34114830 con 0x7fcd34105040 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd39e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34114b30 con 0x7fcd34104680 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34105040 0x7fcd3419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd33400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3410a680 0x7fcd3419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34105040 0x7fcd3419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47552/0 (socket says 172.21.15.94:47552) 2024-06-03T01:04:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd33400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3410a680 0x7fcd3419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59396/0 (socket says 172.21.15.94:59396) 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32000700 1 -- 172.21.15.94:0/3025402938 learned_addr learned my addr 172.21.15.94:0/3025402938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32a00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34104680 0x7fcd3419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32000700 1 -- 172.21.15.94:0/3025402938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34104680 msgr2=0x7fcd3419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32000700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34104680 0x7fcd3419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32000700 1 -- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3410a680 msgr2=0x7fcd3419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.746+0000 7fcd32000700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3410a680 0x7fcd3419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.747+0000 7fcd32000700 1 -- 172.21.15.94:0/3025402938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3419fdd0 con 0x7fcd34105040 2024-06-03T01:04:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.747+0000 7fcd32000700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34105040 0x7fcd3419b300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000bae0 tx=0x7fcd2000bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.747+0000 7fcd38a00700 1 -- 172.21.15.94:0/3025402938 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20026070 con 0x7fcd34105040 2024-06-03T01:04:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.747+0000 7fcd38a00700 1 -- 172.21.15.94:0/3025402938 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd20003d70 con 0x7fcd34105040 2024-06-03T01:04:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.747+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd341a0060 con 0x7fcd34105040 2024-06-03T01:04:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.747+0000 7fcd38a00700 1 -- 172.21.15.94:0/3025402938 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20021410 con 0x7fcd34105040 2024-06-03T01:04:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.747+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd341a7940 con 0x7fcd34105040 2024-06-03T01:04:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.750+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd34069180 con 0x7fcd34105040 2024-06-03T01:04:43.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.751+0000 7fcd38a00700 1 -- 172.21.15.94:0/3025402938 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcd20029070 con 0x7fcd34105040 2024-06-03T01:04:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.752+0000 7fcd38a00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fcd180a2c20 0x7fcd180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.752+0000 7fcd38a00700 1 -- 172.21.15.94:0/3025402938 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcd200cfe90 con 0x7fcd34105040 2024-06-03T01:04:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.752+0000 7fcd32a00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fcd180a2c20 0x7fcd180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.753+0000 7fcd32a00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fcd180a2c20 0x7fcd180a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009be0 tx=0x7fcd24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.754+0000 7fcd38a00700 1 -- 172.21.15.94:0/3025402938 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcd20030080 con 0x7fcd34105040 2024-06-03T01:04:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:43 smithi107 ceph-mon[36622]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:04:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:04:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:04:43.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.982+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcd3402dca0 con 0x7fcd34105040 2024-06-03T01:04:43.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.983+0000 7fcd38a00700 1 -- 172.21.15.94:0/3025402938 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fcd20013070 con 0x7fcd34105040 2024-06-03T01:04:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:04:43.983 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-06-03T01:04:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:04:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-06-03T01:04:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-06-03T01:04:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:04:43.984 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-06-03T01:04:43.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fcd180a2c20 msgr2=0x7fcd180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:43.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fcd180a2c20 0x7fcd180a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009be0 tx=0x7fcd24009260 comp rx=0 tx=0).stop 2024-06-03T01:04:43.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34105040 msgr2=0x7fcd3419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:43.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34105040 0x7fcd3419b300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000bae0 tx=0x7fcd2000bb10 comp rx=0 tx=0).stop 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 shutdown_connections 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fcd180a2c20 0x7fcd180a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd3410a680 0x7fcd3419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd34104680 0x7fcd3419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 --2- 172.21.15.94:0/3025402938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd34105040 0x7fcd3419b300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 >> 172.21.15.94:0/3025402938 conn(0x7fcd34100150 msgr2=0x7fcd34100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 shutdown_connections 2024-06-03T01:04:43.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:43.986+0000 7fcd39e00700 1 -- 172.21.15.94:0/3025402938 wait complete. 2024-06-03T01:04:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:44 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3025402938' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T01:04:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:44 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3025402938' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T01:04:44.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:44 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3025402938' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-03T01:04:44.998 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-03T01:04:44.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-03T01:04:45.456 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:45 smithi107 ceph-mon[36622]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:45 smithi094 ceph-mon[32119]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:45 smithi179 ceph-mon[37041]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:47.590 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:47 smithi094 ceph-mon[32119]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:47.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:47 smithi179 ceph-mon[37041]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:47.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:47 smithi107 ceph-mon[36622]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 -- 172.21.15.94:0/3984629695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c107840 msgr2=0x7f395c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 --2- 172.21.15.94:0/3984629695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c107840 0x7f395c10f3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f395400b0a0 tx=0x7f395400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 -- 172.21.15.94:0/3984629695 shutdown_connections 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 --2- 172.21.15.94:0/3984629695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f395c102740 0x7f395c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 --2- 172.21.15.94:0/3984629695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c107840 0x7f395c10f3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 --2- 172.21.15.94:0/3984629695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c1030f0 0x7f395c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 -- 172.21.15.94:0/3984629695 >> 172.21.15.94:0/3984629695 conn(0x7f395c0795f0 msgr2=0x7f395c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 -- 172.21.15.94:0/3984629695 shutdown_connections 2024-06-03T01:04:47.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.940+0000 7f3963000700 1 -- 172.21.15.94:0/3984629695 wait complete. 2024-06-03T01:04:47.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 Processor -- start 2024-06-03T01:04:47.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 -- start start 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f395c102740 0x7f395c19f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c1030f0 0x7f395c19feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c107840 0x7f395c19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f395c074fa0 con 0x7f395c107840 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f395c074e20 con 0x7f395c102740 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.942+0000 7f3963000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f395c075120 con 0x7f395c1030f0 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c1030f0 0x7f395c19feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3961000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c107840 0x7f395c19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f395be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f395c102740 0x7f395c19f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c1030f0 0x7f395c19feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55816/0 (socket says 172.21.15.94:55816) 2024-06-03T01:04:47.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 -- 172.21.15.94:0/492063074 learned_addr learned my addr 172.21.15.94:0/492063074 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:47.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 -- 172.21.15.94:0/492063074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f395c102740 msgr2=0x7f395c19f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:47.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f395c102740 0x7f395c19f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:47.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 -- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c107840 msgr2=0x7f395c19aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:47.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c107840 0x7f395c19aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:47.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 -- 172.21.15.94:0/492063074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395c19b8f0 con 0x7f395c1030f0 2024-06-03T01:04:47.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.943+0000 7f3953e00700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c1030f0 0x7f395c19feb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f394400e9a0 tx=0x7f394400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:47.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.944+0000 7f3959600700 1 -- 172.21.15.94:0/492063074 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394400ca10 con 0x7f395c1030f0 2024-06-03T01:04:47.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.944+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395c19bbe0 con 0x7f395c1030f0 2024-06-03T01:04:47.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.944+0000 7f3959600700 1 -- 172.21.15.94:0/492063074 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f394400cbb0 con 0x7f395c1030f0 2024-06-03T01:04:47.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.944+0000 7f3959600700 1 -- 172.21.15.94:0/492063074 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944018680 con 0x7f395c1030f0 2024-06-03T01:04:47.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.944+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395c1a7a70 con 0x7f395c1030f0 2024-06-03T01:04:47.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.945+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395c069180 con 0x7f395c1030f0 2024-06-03T01:04:47.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.947+0000 7f3959600700 1 -- 172.21.15.94:0/492063074 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3944010430 con 0x7f395c1030f0 2024-06-03T01:04:47.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.948+0000 7f3959600700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f39400a2c20 0x7f39400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:47.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.948+0000 7f3959600700 1 -- 172.21.15.94:0/492063074 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f39440cee60 con 0x7f395c1030f0 2024-06-03T01:04:47.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.949+0000 7f395be00700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f39400a2c20 0x7f39400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:47.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.949+0000 7f395be00700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f39400a2c20 0x7f39400a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f395c10f5c0 tx=0x7f394c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:47.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:47.953+0000 7f3959600700 1 -- 172.21.15.94:0/492063074 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f394409c1c0 con 0x7f395c1030f0 2024-06-03T01:04:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.172+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f395c1a7c40 con 0x7f395c1030f0 2024-06-03T01:04:48.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.757+0000 7f3959600700 1 -- 172.21.15.94:0/492063074 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f394409bac0 con 0x7f395c1030f0 2024-06-03T01:04:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f39400a2c20 msgr2=0x7f39400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f39400a2c20 0x7f39400a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f395c10f5c0 tx=0x7f394c009260 comp rx=0 tx=0).stop 2024-06-03T01:04:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c1030f0 msgr2=0x7f395c19feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:48.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c1030f0 0x7f395c19feb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f394400e9a0 tx=0x7f394400ecb0 comp rx=0 tx=0).stop 2024-06-03T01:04:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 shutdown_connections 2024-06-03T01:04:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f395c102740 0x7f395c19f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f39400a2c20 0x7f39400a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f395c107840 0x7f395c19aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 --2- 172.21.15.94:0/492063074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f395c1030f0 0x7f395c19feb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.764+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 >> 172.21.15.94:0/492063074 conn(0x7f395c0795f0 msgr2=0x7f395c106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:48.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.765+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 shutdown_connections 2024-06-03T01:04:48.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:48.766+0000 7f3963000700 1 -- 172.21.15.94:0/492063074 wait complete. 2024-06-03T01:04:48.768 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-06-03T01:04:48.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:48 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/492063074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T01:04:48.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:48 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T01:04:48.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:48 smithi094 ceph-mon[32119]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:48 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/492063074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T01:04:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:48 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T01:04:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:48 smithi107 ceph-mon[36622]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:49.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:48 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/492063074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T01:04:49.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:48 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-03T01:04:49.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:48 smithi179 ceph-mon[37041]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:49.697 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-03T01:04:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:49 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T01:04:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:49 smithi107 ceph-mon[36622]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T01:04:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:49 smithi107 ceph-mon[36622]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T01:04:50.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:49 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T01:04:50.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:49 smithi094 ceph-mon[32119]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T01:04:50.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:49 smithi094 ceph-mon[32119]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T01:04:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:49 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-03T01:04:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:49 smithi179 ceph-mon[37041]: osdmap e83: 12 total, 12 up, 12 in 2024-06-03T01:04:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:49 smithi179 ceph-mon[37041]: osdmap e84: 12 total, 12 up, 12 in 2024-06-03T01:04:50.144 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:51.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:50 smithi094 ceph-mon[32119]: pgmap v322: 33 pgs: 3 creating+activating, 9 creating+peering, 18 unknown, 3 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:51.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:50 smithi094 ceph-mon[32119]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T01:04:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:50 smithi179 ceph-mon[37041]: pgmap v322: 33 pgs: 3 creating+activating, 9 creating+peering, 18 unknown, 3 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:51.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:50 smithi179 ceph-mon[37041]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T01:04:51.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:50 smithi107 ceph-mon[36622]: pgmap v322: 33 pgs: 3 creating+activating, 9 creating+peering, 18 unknown, 3 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:51.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:50 smithi107 ceph-mon[36622]: osdmap e85: 12 total, 12 up, 12 in 2024-06-03T01:04:51.996 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:51 smithi094 ceph-mon[32119]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:04:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:51 smithi179 ceph-mon[37041]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:04:52.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:51 smithi107 ceph-mon[36622]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:04:52.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 -- 172.21.15.94:0/1648208790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb8109390 msgr2=0x7f9eb8115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:52.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1648208790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb8109390 0x7f9eb8115830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009f90 tx=0x7f9eb000b330 comp rx=0 tx=0).stop 2024-06-03T01:04:52.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 -- 172.21.15.94:0/1648208790 shutdown_connections 2024-06-03T01:04:52.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1648208790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb8109390 0x7f9eb8115830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:52.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1648208790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eb810d8a0 0x7f9eb810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:52.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1648208790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb81089d0 0x7f9eb8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:52.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 -- 172.21.15.94:0/1648208790 >> 172.21.15.94:0/1648208790 conn(0x7f9eb8078220 msgr2=0x7f9eb81072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:52.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 -- 172.21.15.94:0/1648208790 shutdown_connections 2024-06-03T01:04:52.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.514+0000 7f9ebd600700 1 -- 172.21.15.94:0/1648208790 wait complete. 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 Processor -- start 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 -- start start 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb81089d0 0x7f9eb819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eb8109390 0x7f9eb819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb810d8a0 0x7f9eb81a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8118d20 con 0x7f9eb810d8a0 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8118ba0 con 0x7f9eb81089d0 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.515+0000 7f9ebd600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8118ea0 con 0x7f9eb8109390 2024-06-03T01:04:52.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9ebcc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb810d8a0 0x7f9eb81a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:52.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb81089d0 0x7f9eb819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:52.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb81089d0 0x7f9eb819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:58840/0 (socket says 172.21.15.94:58840) 2024-06-03T01:04:52.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 -- 172.21.15.94:0/1123989944 learned_addr learned my addr 172.21.15.94:0/1123989944 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:52.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7400700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eb8109390 0x7f9eb819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 -- 172.21.15.94:0/1123989944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eb8109390 msgr2=0x7f9eb819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eb8109390 0x7f9eb819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 -- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb810d8a0 msgr2=0x7f9eb81a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb810d8a0 0x7f9eb81a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9eb7e00700 1 -- 172.21.15.94:0/1123989944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb81a4110 con 0x7f9eb81089d0 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.516+0000 7f9ebcc00700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb810d8a0 0x7f9eb81a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.517+0000 7f9eb7e00700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb81089d0 0x7f9eb819f150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800e9a0 tx=0x7f9ea800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.517+0000 7f9eb4c00700 1 -- 172.21.15.94:0/1123989944 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800ca10 con 0x7f9eb81089d0 2024-06-03T01:04:52.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.517+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb81a4400 con 0x7f9eb81089d0 2024-06-03T01:04:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.517+0000 7f9eb4c00700 1 -- 172.21.15.94:0/1123989944 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea800cbb0 con 0x7f9eb81089d0 2024-06-03T01:04:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.517+0000 7f9eb4c00700 1 -- 172.21.15.94:0/1123989944 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8018680 con 0x7f9eb81089d0 2024-06-03T01:04:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.517+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb81abc80 con 0x7f9eb81089d0 2024-06-03T01:04:52.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.518+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb8069180 con 0x7f9eb81089d0 2024-06-03T01:04:52.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.520+0000 7f9eb4c00700 1 -- 172.21.15.94:0/1123989944 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ea8010430 con 0x7f9eb81089d0 2024-06-03T01:04:52.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.522+0000 7f9eb4c00700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e9c0a2c20 0x7f9e9c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:52.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.522+0000 7f9eb4c00700 1 -- 172.21.15.94:0/1123989944 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f9ea80cf2e0 con 0x7f9eb81089d0 2024-06-03T01:04:52.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.522+0000 7f9eb7400700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e9c0a2c20 0x7f9e9c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:52.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.522+0000 7f9eb7400700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e9c0a2c20 0x7f9e9c0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009750 tx=0x7f9ea4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:52.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.524+0000 7f9eb4c00700 1 -- 172.21.15.94:0/1123989944 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ea809c4d0 con 0x7f9eb81089d0 2024-06-03T01:04:52.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:52.751+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f9eb81a0460 con 0x7f9eb81089d0 2024-06-03T01:04:53.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.342+0000 7f9eb4c00700 1 -- 172.21.15.94:0/1123989944 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) ==== 171+0+0 (secure 0 0 0) 0x7f9ea809bdd0 con 0x7f9eb81089d0 2024-06-03T01:04:53.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.348+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e9c0a2c20 msgr2=0x7f9e9c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.348+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e9c0a2c20 0x7f9e9c0a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009750 tx=0x7f9ea4009310 comp rx=0 tx=0).stop 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.348+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb81089d0 msgr2=0x7f9eb819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.348+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb81089d0 0x7f9eb819f150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800e9a0 tx=0x7f9ea800ecb0 comp rx=0 tx=0).stop 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.349+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 shutdown_connections 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.349+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eb81089d0 0x7f9eb819f150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.349+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9e9c0a2c20 0x7f9e9c0a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.349+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eb810d8a0 0x7f9eb81a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.349+0000 7f9ebd600700 1 --2- 172.21.15.94:0/1123989944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eb8109390 0x7f9eb819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:53.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.349+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 >> 172.21.15.94:0/1123989944 conn(0x7f9eb8078220 msgr2=0x7f9eb8078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.349+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 shutdown_connections 2024-06-03T01:04:53.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:53.350+0000 7f9ebd600700 1 -- 172.21.15.94:0/1123989944 wait complete. 2024-06-03T01:04:53.353 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-06-03T01:04:53.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:53 smithi094 ceph-mon[32119]: pgmap v324: 33 pgs: 3 creating+activating, 9 creating+peering, 2 unknown, 19 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:53.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:53 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1123989944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T01:04:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:53 smithi179 ceph-mon[37041]: pgmap v324: 33 pgs: 3 creating+activating, 9 creating+peering, 2 unknown, 19 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:53 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1123989944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T01:04:53.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:53 smithi107 ceph-mon[36622]: pgmap v324: 33 pgs: 3 creating+activating, 9 creating+peering, 2 unknown, 19 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:53.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:53 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1123989944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T01:04:54.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-03T01:04:54.542 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:54 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T01:04:54.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:54 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T01:04:54.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:54 smithi094 ceph-mon[32119]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T01:04:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:54 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T01:04:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:54 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T01:04:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:54 smithi179 ceph-mon[37041]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T01:04:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:54 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-03T01:04:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:54 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-03T01:04:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:54 smithi107 ceph-mon[36622]: osdmap e86: 12 total, 12 up, 12 in 2024-06-03T01:04:54.916 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:55.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:55 smithi094 ceph-mon[32119]: pgmap v326: 97 pgs: 57 unknown, 3 creating+activating, 16 creating+peering, 21 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:55.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:55 smithi094 ceph-mon[32119]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T01:04:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:55 smithi179 ceph-mon[37041]: pgmap v326: 97 pgs: 57 unknown, 3 creating+activating, 16 creating+peering, 21 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:55 smithi179 ceph-mon[37041]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T01:04:55.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:55 smithi107 ceph-mon[36622]: pgmap v326: 97 pgs: 57 unknown, 3 creating+activating, 16 creating+peering, 21 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:55.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:55 smithi107 ceph-mon[36622]: osdmap e87: 12 total, 12 up, 12 in 2024-06-03T01:04:56.701 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:56 smithi107 ceph-mon[36622]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T01:04:56.701 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:56 smithi107 ceph-mon[36622]: pgmap v329: 97 pgs: 24 unknown, 21 creating+peering, 52 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:56.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:56 smithi094 ceph-mon[32119]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T01:04:56.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:56 smithi094 ceph-mon[32119]: pgmap v329: 97 pgs: 24 unknown, 21 creating+peering, 52 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:56.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:56 smithi179 ceph-mon[37041]: osdmap e88: 12 total, 12 up, 12 in 2024-06-03T01:04:56.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:56 smithi179 ceph-mon[37041]: pgmap v329: 97 pgs: 24 unknown, 21 creating+peering, 52 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 -- 172.21.15.94:0/1243681850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 msgr2=0x7f9eec113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/1243681850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec113120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009990 tx=0x7f9ed8009ca0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 -- 172.21.15.94:0/1243681850 shutdown_connections 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/1243681850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec113660 0x7f9eec115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/1243681850 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eec0746c0 0x7f9eec074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/1243681850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec113120 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 -- 172.21.15.94:0/1243681850 >> 172.21.15.94:0/1243681850 conn(0x7f9eec0795f0 msgr2=0x7f9eec079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:57.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 -- 172.21.15.94:0/1243681850 shutdown_connections 2024-06-03T01:04:57.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.338+0000 7f9ef3e00700 1 -- 172.21.15.94:0/1243681850 wait complete. 2024-06-03T01:04:57.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.340+0000 7f9ef3e00700 1 Processor -- start 2024-06-03T01:04:57.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.340+0000 7f9ef3e00700 1 -- start start 2024-06-03T01:04:57.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.340+0000 7f9ef3e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec0746c0 0x7f9eec19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:57.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.340+0000 7f9ef3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.340+0000 7f9ef3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eec113660 0x7f9eec1a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.340+0000 7f9ef3e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec118df0 con 0x7f9eec113660 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef3e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec118c70 con 0x7f9eec074fe0 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef3e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec118f70 con 0x7f9eec0746c0 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec19f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41332/0 (socket says 172.21.15.94:41332) 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef1400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec0746c0 0x7f9eec19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 -- 172.21.15.94:0/3052243779 learned_addr learned my addr 172.21.15.94:0/3052243779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef1e00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eec113660 0x7f9eec1a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 -- 172.21.15.94:0/3052243779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec0746c0 msgr2=0x7f9eec19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:57.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec0746c0 0x7f9eec19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 -- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eec113660 msgr2=0x7f9eec1a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:57.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eec113660 0x7f9eec1a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.341+0000 7f9ef0a00700 1 -- 172.21.15.94:0/3052243779 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec1a43e0 con 0x7f9eec074fe0 2024-06-03T01:04:57.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.342+0000 7f9ef0a00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec19f910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed800ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:57.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.342+0000 7f9eea000700 1 -- 172.21.15.94:0/3052243779 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8027070 con 0x7f9eec074fe0 2024-06-03T01:04:57.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.342+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec1a4670 con 0x7f9eec074fe0 2024-06-03T01:04:57.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.342+0000 7f9eea000700 1 -- 172.21.15.94:0/3052243779 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed8003d70 con 0x7f9eec074fe0 2024-06-03T01:04:57.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.342+0000 7f9eea000700 1 -- 172.21.15.94:0/3052243779 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800f460 con 0x7f9eec074fe0 2024-06-03T01:04:57.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.342+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec1abf50 con 0x7f9eec074fe0 2024-06-03T01:04:57.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.343+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec075db0 con 0x7f9eec074fe0 2024-06-03T01:04:57.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.345+0000 7f9eea000700 1 -- 172.21.15.94:0/3052243779 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ed802a070 con 0x7f9eec074fe0 2024-06-03T01:04:57.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.346+0000 7f9eea000700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9ed00a2c20 0x7f9ed00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:04:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.347+0000 7f9eea000700 1 -- 172.21.15.94:0/3052243779 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7f9ed80cfe20 con 0x7f9eec074fe0 2024-06-03T01:04:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.347+0000 7f9ef1400700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9ed00a2c20 0x7f9ed00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:04:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.347+0000 7f9ef1400700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9ed00a2c20 0x7f9ed00a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00f540 tx=0x7f9edc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:04:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.355+0000 7f9eea000700 1 -- 172.21.15.94:0/3052243779 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9ed805e850 con 0x7f9eec074fe0 2024-06-03T01:04:57.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.627+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9eec1ac120 con 0x7f9eec074fe0 2024-06-03T01:04:57.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.688+0000 7f9eea000700 1 -- 172.21.15.94:0/3052243779 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f9ed8013070 con 0x7f9eec074fe0 2024-06-03T01:04:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.694+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9ed00a2c20 msgr2=0x7f9ed00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.694+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9ed00a2c20 0x7f9ed00a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00f540 tx=0x7f9edc00b040 comp rx=0 tx=0).stop 2024-06-03T01:04:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.694+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 msgr2=0x7f9eec19f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:04:57.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.694+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec19f910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed800ba90 comp rx=0 tx=0).stop 2024-06-03T01:04:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.695+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 shutdown_connections 2024-06-03T01:04:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.695+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f9ed00a2c20 0x7f9ed00a50e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.695+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eec113660 0x7f9eec1a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.695+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9eec0746c0 0x7f9eec19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.695+0000 7f9ef3e00700 1 --2- 172.21.15.94:0/3052243779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9eec074fe0 0x7f9eec19f910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:04:57.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.695+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 >> 172.21.15.94:0/3052243779 conn(0x7f9eec0795f0 msgr2=0x7f9eec112420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:04:57.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.696+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 shutdown_connections 2024-06-03T01:04:57.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:04:57.697+0000 7f9ef3e00700 1 -- 172.21.15.94:0/3052243779 wait complete. 2024-06-03T01:04:57.702 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-06-03T01:04:58.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:57 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3052243779' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T01:04:58.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:57 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T01:04:58.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:57 smithi107 ceph-mon[36622]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-03T01:04:58.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:57 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3052243779' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T01:04:58.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:57 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T01:04:58.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:57 smithi094 ceph-mon[32119]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-03T01:04:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:57 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3052243779' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T01:04:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:57 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-03T01:04:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:57 smithi179 ceph-mon[37041]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-03T01:04:58.604 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-03T01:04:58.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-03T01:04:58.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T01:04:58.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T01:04:58.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-03T01:04:58.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-03T01:04:58.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: pgmap v331: 97 pgs: 21 creating+peering, 76 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:58.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:04:58.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] up:active 2024-06-03T01:04:58.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:58 smithi094 ceph-mon[32119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:04:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T01:04:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T01:04:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-03T01:04:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-03T01:04:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: pgmap v331: 97 pgs: 21 creating+peering, 76 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:04:59.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] up:active 2024-06-03T01:04:59.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:58 smithi107 ceph-mon[36622]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:04:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: osdmap e89: 12 total, 12 up, 12 in 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: pgmap v331: 97 pgs: 21 creating+peering, 76 active+clean; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] up:active 2024-06-03T01:04:59.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:58 smithi179 ceph-mon[37041]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:04:59.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:59 smithi094 ceph-mon[32119]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T01:04:59.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:04:59 smithi094 ceph-mon[32119]: Cluster is now healthy 2024-06-03T01:05:00.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:59 smithi107 ceph-mon[36622]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T01:05:00.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:04:59 smithi107 ceph-mon[36622]: Cluster is now healthy 2024-06-03T01:05:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:59 smithi179 ceph-mon[37041]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-03T01:05:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:04:59 smithi179 ceph-mon[37041]: Cluster is now healthy 2024-06-03T01:05:00.705 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:05:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: 2024-06-03T01:05:00.361+0000 7fc5fe11d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-03T01:05:00.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:00 smithi094 ceph-mon[32119]: pgmap v332: 97 pgs: 97 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-06-03T01:05:01.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:00 smithi107 ceph-mon[36622]: pgmap v332: 97 pgs: 97 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-06-03T01:05:01.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:00 smithi179 ceph-mon[37041]: pgmap v332: 97 pgs: 97 active+clean; 577 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 -- 172.21.15.94:0/836934886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c10eac0 msgr2=0x7fdc0c10eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 --2- 172.21.15.94:0/836934886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c10eac0 0x7fdc0c10eea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0098f0 tx=0x7fdbfc009c00 comp rx=0 tx=0).stop 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 -- 172.21.15.94:0/836934886 shutdown_connections 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 --2- 172.21.15.94:0/836934886 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0c109480 0x7fdc0c115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 --2- 172.21.15.94:0/836934886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c10eac0 0x7fdc0c10eea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 --2- 172.21.15.94:0/836934886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108ac0 0x7fdc0c108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 -- 172.21.15.94:0/836934886 >> 172.21.15.94:0/836934886 conn(0x7fdc0c075c60 msgr2=0x7fdc0c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 -- 172.21.15.94:0/836934886 shutdown_connections 2024-06-03T01:05:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.371+0000 7fdc10a00700 1 -- 172.21.15.94:0/836934886 wait complete. 2024-06-03T01:05:01.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.373+0000 7fdc10a00700 1 Processor -- start 2024-06-03T01:05:01.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.373+0000 7fdc10a00700 1 -- start start 2024-06-03T01:05:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc10a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c108ac0 0x7fdc0c19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc10a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0c109480 0x7fdc0c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc10a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c10eac0 0x7fdc0c1a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc10a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c118e40 con 0x7fdc0c109480 2024-06-03T01:05:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc10a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c118cc0 con 0x7fdc0c10eac0 2024-06-03T01:05:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc10a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c118fc0 con 0x7fdc0c108ac0 2024-06-03T01:05:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc09400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0c109480 0x7fdc0c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc09e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c108ac0 0x7fdc0c19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc09400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0c109480 0x7fdc0c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59504/0 (socket says 172.21.15.94:59504) 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc09400700 1 -- 172.21.15.94:0/1366315388 learned_addr learned my addr 172.21.15.94:0/1366315388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.374+0000 7fdc0a800700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c10eac0 0x7fdc0c1a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc0a800700 1 -- 172.21.15.94:0/1366315388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c108ac0 msgr2=0x7fdc0c19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc0a800700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c108ac0 0x7fdc0c19f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc0a800700 1 -- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0c109480 msgr2=0x7fdc0c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc0a800700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0c109480 0x7fdc0c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc0a800700 1 -- 172.21.15.94:0/1366315388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c1a42e0 con 0x7fdc0c10eac0 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc09e00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c108ac0 0x7fdc0c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc0a800700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c10eac0 0x7fdc0c1a3ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400efc0 tx=0x7fdc0400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc02a00700 1 -- 172.21.15.94:0/1366315388 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc040092e0 con 0x7fdc0c10eac0 2024-06-03T01:05:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c1a4570 con 0x7fdc0c10eac0 2024-06-03T01:05:01.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc02a00700 1 -- 172.21.15.94:0/1366315388 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc04004bb0 con 0x7fdc0c10eac0 2024-06-03T01:05:01.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc02a00700 1 -- 172.21.15.94:0/1366315388 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04004280 con 0x7fdc0c10eac0 2024-06-03T01:05:01.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.375+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c1abe50 con 0x7fdc0c10eac0 2024-06-03T01:05:01.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.376+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0c069180 con 0x7fdc0c10eac0 2024-06-03T01:05:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.378+0000 7fdc02a00700 1 -- 172.21.15.94:0/1366315388 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc0401b030 con 0x7fdc0c10eac0 2024-06-03T01:05:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.380+0000 7fdc02a00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdbf00a2ca0 0x7fdbf00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.380+0000 7fdc02a00700 1 -- 172.21.15.94:0/1366315388 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fdc040ce8e0 con 0x7fdc0c10eac0 2024-06-03T01:05:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.380+0000 7fdc09e00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdbf00a2ca0 0x7fdbf00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.380+0000 7fdc09e00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdbf00a2ca0 0x7fdbf00a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009810 tx=0x7fdbfc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:01.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.384+0000 7fdc02a00700 1 -- 172.21.15.94:0/1366315388 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdc04022020 con 0x7fdc0c10eac0 2024-06-03T01:05:01.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.617+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fdc0c02dca0 con 0x7fdc0c10eac0 2024-06-03T01:05:01.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.719+0000 7fdc02a00700 1 -- 172.21.15.94:0/1366315388 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v90) ==== 157+0+0 (secure 0 0 0) 0x7fdc04010040 con 0x7fdc0c10eac0 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdbf00a2ca0 msgr2=0x7fdbf00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdbf00a2ca0 0x7fdbf00a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009810 tx=0x7fdbfc019040 comp rx=0 tx=0).stop 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c10eac0 msgr2=0x7fdc0c1a3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c10eac0 0x7fdc0c1a3ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400efc0 tx=0x7fdc0400c3a0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 shutdown_connections 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c10eac0 0x7fdc0c1a3ba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdbf00a2ca0 0x7fdbf00a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc0c108ac0 0x7fdc0c19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 --2- 172.21.15.94:0/1366315388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc0c109480 0x7fdc0c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:01.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.725+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 >> 172.21.15.94:0/1366315388 conn(0x7fdc0c075c60 msgr2=0x7fdc0c076280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:01.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.726+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 shutdown_connections 2024-06-03T01:05:01.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:01.726+0000 7fdc10a00700 1 -- 172.21.15.94:0/1366315388 wait complete. 2024-06-03T01:05:02.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:01 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1366315388' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T01:05:02.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:01 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T01:05:02.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:01 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1366315388' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T01:05:02.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:01 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T01:05:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:01 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1366315388' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T01:05:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:01 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-03T01:05:02.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-03T01:05:02.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:02 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T01:05:02.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:02 smithi094 ceph-mon[32119]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T01:05:02.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:02 smithi094 ceph-mon[32119]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-06-03T01:05:03.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:02 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T01:05:03.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:02 smithi107 ceph-mon[36622]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T01:05:03.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:02 smithi107 ceph-mon[36622]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-06-03T01:05:03.069 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:03.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:02 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-03T01:05:03.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:02 smithi179 ceph-mon[37041]: osdmap e90: 12 total, 12 up, 12 in 2024-06-03T01:05:03.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:02 smithi179 ceph-mon[37041]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 -- 172.21.15.94:0/1759485593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c06a3d0 msgr2=0x7fea8c06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 --2- 172.21.15.94:0/1759485593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c06a3d0 0x7fea8c06a7b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fea800099c0 tx=0x7fea80009cd0 comp rx=0 tx=0).stop 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 -- 172.21.15.94:0/1759485593 shutdown_connections 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 --2- 172.21.15.94:0/1759485593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c107850 0x7fea8c10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 --2- 172.21.15.94:0/1759485593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c06a3d0 0x7fea8c06a7b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 --2- 172.21.15.94:0/1759485593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea8c06ad80 0x7fea8c107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 -- 172.21.15.94:0/1759485593 >> 172.21.15.94:0/1759485593 conn(0x7fea8c078330 msgr2=0x7fea8c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:05.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 -- 172.21.15.94:0/1759485593 shutdown_connections 2024-06-03T01:05:05.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.347+0000 7fea94200700 1 -- 172.21.15.94:0/1759485593 wait complete. 2024-06-03T01:05:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.348+0000 7fea94200700 1 Processor -- start 2024-06-03T01:05:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea94200700 1 -- start start 2024-06-03T01:05:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea94200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c06a3d0 0x7fea8c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea94200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea8c06ad80 0x7fea8c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:05.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea94200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c107850 0x7fea8c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea94200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1127c0 con 0x7fea8c107850 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea94200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c112640 con 0x7fea8c06ad80 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea94200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c112940 con 0x7fea8c06a3d0 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea91800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c06a3d0 0x7fea8c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea92200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c107850 0x7fea8c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea91800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c06a3d0 0x7fea8c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:54704/0 (socket says 172.21.15.94:54704) 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.349+0000 7fea91800700 1 -- 172.21.15.94:0/2556323791 learned_addr learned my addr 172.21.15.94:0/2556323791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea90e00700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea8c06ad80 0x7fea8c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea91800700 1 -- 172.21.15.94:0/2556323791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea8c06ad80 msgr2=0x7fea8c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea91800700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea8c06ad80 0x7fea8c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea91800700 1 -- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c107850 msgr2=0x7fea8c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:05.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea91800700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c107850 0x7fea8c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:05.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea91800700 1 -- 172.21.15.94:0/2556323791 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c1990a0 con 0x7fea8c06a3d0 2024-06-03T01:05:05.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea91800700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c06a3d0 0x7fea8c19e780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea80006010 tx=0x7fea80003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:05.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.350+0000 7fea8a000700 1 -- 172.21.15.94:0/2556323791 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80003680 con 0x7fea8c06a3d0 2024-06-03T01:05:05.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.351+0000 7fea8a000700 1 -- 172.21.15.94:0/2556323791 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea800055a0 con 0x7fea8c06a3d0 2024-06-03T01:05:05.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.351+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c199330 con 0x7fea8c06a3d0 2024-06-03T01:05:05.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.351+0000 7fea8a000700 1 -- 172.21.15.94:0/2556323791 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8000fe00 con 0x7fea8c06a3d0 2024-06-03T01:05:05.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.351+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c1a5650 con 0x7fea8c06a3d0 2024-06-03T01:05:05.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.352+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8c050a00 con 0x7fea8c06a3d0 2024-06-03T01:05:05.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.354+0000 7fea8a000700 1 -- 172.21.15.94:0/2556323791 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fea8000f4f0 con 0x7fea8c06a3d0 2024-06-03T01:05:05.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.355+0000 7fea8a000700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea740a2ee0 0x7fea740a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:05.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.355+0000 7fea8a000700 1 -- 172.21.15.94:0/2556323791 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fea800cfcc0 con 0x7fea8c06a3d0 2024-06-03T01:05:05.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.355+0000 7fea90e00700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea740a2ee0 0x7fea740a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:05.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.356+0000 7fea90e00700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea740a2ee0 0x7fea740a53a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea78009680 tx=0x7fea78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:05.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.360+0000 7fea8a000700 1 -- 172.21.15.94:0/2556323791 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea800d6080 con 0x7fea8c06a3d0 2024-06-03T01:05:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:05.585+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fea8c199e70 con 0x7fea8c06a3d0 2024-06-03T01:05:05.586 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:05 smithi094 ceph-mon[32119]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T01:05:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:05 smithi179 ceph-mon[37041]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T01:05:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:05 smithi107 ceph-mon[36622]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-03T01:05:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:06 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2556323791' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T01:05:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:06 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T01:05:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:06 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2556323791' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T01:05:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:06 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T01:05:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:06 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2556323791' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T01:05:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:06 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-03T01:05:07.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.358+0000 7fea8a000700 1 -- 172.21.15.94:0/2556323791 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v92) ==== 228+0+0 (secure 0 0 0) 0x7fea8c199e70 con 0x7fea8c06a3d0 2024-06-03T01:05:07.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.365+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea740a2ee0 msgr2=0x7fea740a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:07.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.365+0000 7fea94200700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea740a2ee0 0x7fea740a53a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fea78009680 tx=0x7fea78009260 comp rx=0 tx=0).stop 2024-06-03T01:05:07.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.365+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c06a3d0 msgr2=0x7fea8c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:07.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.365+0000 7fea94200700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c06a3d0 0x7fea8c19e780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea80006010 tx=0x7fea80003a90 comp rx=0 tx=0).stop 2024-06-03T01:05:07.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.365+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 shutdown_connections 2024-06-03T01:05:07.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.365+0000 7fea94200700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fea740a2ee0 0x7fea740a53a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:07.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.366+0000 7fea94200700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea8c107850 0x7fea8c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:07.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.366+0000 7fea94200700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea8c06a3d0 0x7fea8c19e780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:07.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.366+0000 7fea94200700 1 --2- 172.21.15.94:0/2556323791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea8c06ad80 0x7fea8c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:07.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.366+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 >> 172.21.15.94:0/2556323791 conn(0x7fea8c078330 msgr2=0x7fea8c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:07.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.366+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 shutdown_connections 2024-06-03T01:05:07.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:07.368+0000 7fea94200700 1 -- 172.21.15.94:0/2556323791 wait complete. 2024-06-03T01:05:07.372 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data_ec' created 2024-06-03T01:05:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:07 smithi094 ceph-mon[32119]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-06-03T01:05:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:07 smithi094 ceph-mon[32119]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T01:05:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:07 smithi179 ceph-mon[37041]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-06-03T01:05:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:07 smithi179 ceph-mon[37041]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T01:05:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:07 smithi107 ceph-mon[36622]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-06-03T01:05:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:07 smithi107 ceph-mon[36622]: osdmap e91: 12 total, 12 up, 12 in 2024-06-03T01:05:08.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-03T01:05:08.695 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:08 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T01:05:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:08 smithi107 ceph-mon[36622]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T01:05:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:08 smithi107 ceph-mon[36622]: pgmap v339: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 0 op/s 2024-06-03T01:05:08.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:08 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T01:05:08.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:08 smithi094 ceph-mon[32119]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T01:05:08.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:08 smithi094 ceph-mon[32119]: pgmap v339: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 0 op/s 2024-06-03T01:05:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:08 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-03T01:05:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:08 smithi179 ceph-mon[37041]: osdmap e92: 12 total, 12 up, 12 in 2024-06-03T01:05:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:08 smithi179 ceph-mon[37041]: pgmap v339: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 0 op/s 2024-06-03T01:05:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:09 smithi094 ceph-mon[32119]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:05:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:09 smithi094 ceph-mon[32119]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T01:05:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:09 smithi107 ceph-mon[36622]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:05:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:09 smithi107 ceph-mon[36622]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T01:05:09.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:09 smithi179 ceph-mon[37041]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-03T01:05:09.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:09 smithi179 ceph-mon[37041]: osdmap e93: 12 total, 12 up, 12 in 2024-06-03T01:05:10.570 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:10 smithi094 ceph-mon[32119]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T01:05:10.571 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:10 smithi094 ceph-mon[32119]: pgmap v342: 129 pgs: 14 unknown, 115 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:10 smithi107 ceph-mon[36622]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T01:05:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:10 smithi107 ceph-mon[36622]: pgmap v342: 129 pgs: 14 unknown, 115 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:10 smithi179 ceph-mon[37041]: osdmap e94: 12 total, 12 up, 12 in 2024-06-03T01:05:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:10 smithi179 ceph-mon[37041]: pgmap v342: 129 pgs: 14 unknown, 115 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 -- 172.21.15.94:0/3387001528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0107840 msgr2=0x7f73f010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 --2- 172.21.15.94:0/3387001528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0107840 0x7f73f010f3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b0a0 tx=0x7f73e800b3b0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 -- 172.21.15.94:0/3387001528 shutdown_connections 2024-06-03T01:05:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 --2- 172.21.15.94:0/3387001528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0107840 0x7f73f010f3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 --2- 172.21.15.94:0/3387001528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73f0102740 0x7f73f0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 --2- 172.21.15.94:0/3387001528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73f01030f0 0x7f73f0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 -- 172.21.15.94:0/3387001528 >> 172.21.15.94:0/3387001528 conn(0x7f73f00795f0 msgr2=0x7f73f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 -- 172.21.15.94:0/3387001528 shutdown_connections 2024-06-03T01:05:11.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.123+0000 7f73f7000700 1 -- 172.21.15.94:0/3387001528 wait complete. 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.124+0000 7f73f7000700 1 Processor -- start 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f7000700 1 -- start start 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f7000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0102740 0x7f73f019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f7000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73f01030f0 0x7f73f019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f7000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73f0107840 0x7f73f019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f7000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01127b0 con 0x7f73f0107840 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f7000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0112630 con 0x7f73f01030f0 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f7000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0112930 con 0x7f73f0102740 2024-06-03T01:05:11.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f5000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73f0107840 0x7f73f019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73efe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0102740 0x7f73f019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73ef400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73f01030f0 0x7f73f019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f5000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73f0107840 0x7f73f019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54456/0 (socket says 172.21.15.94:54456) 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73f5000700 1 -- 172.21.15.94:0/1850066260 learned_addr learned my addr 172.21.15.94:0/1850066260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.125+0000 7f73efe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0102740 0x7f73f019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:54730/0 (socket says 172.21.15.94:54730) 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73efe00700 1 -- 172.21.15.94:0/1850066260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73f01030f0 msgr2=0x7f73f019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73efe00700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73f01030f0 0x7f73f019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73efe00700 1 -- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73f0107840 msgr2=0x7f73f019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73efe00700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73f0107840 0x7f73f019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73efe00700 1 -- 172.21.15.94:0/1850066260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f019fdd0 con 0x7f73f0102740 2024-06-03T01:05:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73ef400700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73f01030f0 0x7f73f019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73efe00700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0102740 0x7f73f019adc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e000e9f0 tx=0x7f73e000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73ecc00700 1 -- 172.21.15.94:0/1850066260 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e000cb70 con 0x7f73f0102740 2024-06-03T01:05:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.126+0000 7f73ecc00700 1 -- 172.21.15.94:0/1850066260 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e000ee90 con 0x7f73f0102740 2024-06-03T01:05:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.127+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f01a00c0 con 0x7f73f0102740 2024-06-03T01:05:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.127+0000 7f73ecc00700 1 -- 172.21.15.94:0/1850066260 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0003dc0 con 0x7f73f0102740 2024-06-03T01:05:11.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.127+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f01a7940 con 0x7f73f0102740 2024-06-03T01:05:11.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.128+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f0069180 con 0x7f73f0102740 2024-06-03T01:05:11.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.130+0000 7f73ecc00700 1 -- 172.21.15.94:0/1850066260 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f73e0003f80 con 0x7f73f0102740 2024-06-03T01:05:11.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.131+0000 7f73ecc00700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73d40a2ca0 0x7f73d40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:11.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.131+0000 7f73ecc00700 1 -- 172.21.15.94:0/1850066260 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f73e00cf430 con 0x7f73f0102740 2024-06-03T01:05:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.131+0000 7f73ef400700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73d40a2ca0 0x7f73d40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.132+0000 7f73ef400700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73d40a2ca0 0x7f73d40a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73f010f770 tx=0x7f73dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.133+0000 7f73ecc00700 1 -- 172.21.15.94:0/1850066260 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f73e009c1a0 con 0x7f73f0102740 2024-06-03T01:05:11.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.358+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f73f019c120 con 0x7f73f0102740 2024-06-03T01:05:11.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.409+0000 7f73ecc00700 1 -- 172.21.15.94:0/1850066260 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v95) ==== 169+0+0 (secure 0 0 0) 0x7f73e009baa0 con 0x7f73f0102740 2024-06-03T01:05:11.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73d40a2ca0 msgr2=0x7f73d40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:11.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73d40a2ca0 0x7f73d40a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73f010f770 tx=0x7f73dc009260 comp rx=0 tx=0).stop 2024-06-03T01:05:11.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0102740 msgr2=0x7f73f019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:11.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0102740 0x7f73f019adc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e000e9f0 tx=0x7f73e000ed00 comp rx=0 tx=0).stop 2024-06-03T01:05:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 shutdown_connections 2024-06-03T01:05:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73d40a2ca0 0x7f73d40a5160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73f0107840 0x7f73f019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73f0102740 0x7f73f019adc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 --2- 172.21.15.94:0/1850066260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73f01030f0 0x7f73f019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.413+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 >> 172.21.15.94:0/1850066260 conn(0x7f73f00795f0 msgr2=0x7f73f0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.414+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 shutdown_connections 2024-06-03T01:05:11.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:11.414+0000 7f73f7000700 1 -- 172.21.15.94:0/1850066260 wait complete. 2024-06-03T01:05:11.415 INFO:teuthology.orchestra.run.smithi094.stderr:set pool 4 allow_ec_overwrites to true 2024-06-03T01:05:11.783 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:11 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1850066260' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T01:05:11.783 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:11 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T01:05:11.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:11 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1850066260' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T01:05:11.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:11 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T01:05:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:11 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1850066260' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T01:05:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:11 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-03T01:05:12.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-03T01:05:12.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:12 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T01:05:12.414 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:12 smithi094 ceph-mon[32119]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T01:05:12.664 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:12 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T01:05:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:12 smithi107 ceph-mon[36622]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T01:05:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:12 smithi107 ceph-mon[36622]: pgmap v344: 129 pgs: 2 unknown, 127 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:12.813 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:12 smithi094 ceph-mon[32119]: pgmap v344: 129 pgs: 2 unknown, 127 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:12 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-03T01:05:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:12 smithi179 ceph-mon[37041]: osdmap e95: 12 total, 12 up, 12 in 2024-06-03T01:05:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:12 smithi179 ceph-mon[37041]: pgmap v344: 129 pgs: 2 unknown, 127 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:14.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 -- 172.21.15.94:0/923411347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc1045b0 msgr2=0x7fa5fc104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:14.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 --2- 172.21.15.94:0/923411347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc1045b0 0x7fa5fc104a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40099c0 tx=0x7fa5e4009cd0 comp rx=0 tx=0).stop 2024-06-03T01:05:14.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 -- 172.21.15.94:0/923411347 shutdown_connections 2024-06-03T01:05:14.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 --2- 172.21.15.94:0/923411347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5fc10a660 0x7fa5fc10aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:14.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 --2- 172.21.15.94:0/923411347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5fc104f70 0x7fa5fc111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:14.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 --2- 172.21.15.94:0/923411347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc1045b0 0x7fa5fc104a30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:14.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 -- 172.21.15.94:0/923411347 >> 172.21.15.94:0/923411347 conn(0x7fa5fc1000c0 msgr2=0x7fa5fc1024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:14.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.945+0000 7fa5fb200700 1 -- 172.21.15.94:0/923411347 shutdown_connections 2024-06-03T01:05:14.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.946+0000 7fa5fb200700 1 -- 172.21.15.94:0/923411347 wait complete. 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.947+0000 7fa5fb200700 1 Processor -- start 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.947+0000 7fa5fb200700 1 -- start start 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fb200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5fc1045b0 0x7fa5fc19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fb200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc104f70 0x7fa5fc19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fb200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5fc10a660 0x7fa5fc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fb200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc114960 con 0x7fa5fc104f70 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fb200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1147e0 con 0x7fa5fc10a660 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fb200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc114ae0 con 0x7fa5fc1045b0 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc104f70 0x7fa5fc19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:14.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc104f70 0x7fa5fc19b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33548/0 (socket says 172.21.15.94:33548) 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f9e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5fc1045b0 0x7fa5fc19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 -- 172.21.15.94:0/3463631951 learned_addr learned my addr 172.21.15.94:0/3463631951 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fa800700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5fc10a660 0x7fa5fc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 -- 172.21.15.94:0/3463631951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5fc1045b0 msgr2=0x7fa5fc19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5fc1045b0 0x7fa5fc19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 -- 172.21.15.94:0/3463631951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5fc10a660 msgr2=0x7fa5fc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5fc10a660 0x7fa5fc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f3e00700 1 -- 172.21.15.94:0/3463631951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc19fd60 con 0x7fa5fc104f70 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5f9e00700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5fc1045b0 0x7fa5fc19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.948+0000 7fa5fa800700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5fc10a660 0x7fa5fc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:14.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.949+0000 7fa5f3e00700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc104f70 0x7fa5fc19b290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4005910 tx=0x7fa5e400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.949+0000 7fa5f2a00700 1 -- 172.21.15.94:0/3463631951 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4027070 con 0x7fa5fc104f70 2024-06-03T01:05:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.949+0000 7fa5f2a00700 1 -- 172.21.15.94:0/3463631951 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e4003680 con 0x7fa5fc104f70 2024-06-03T01:05:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.949+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc19fff0 con 0x7fa5fc104f70 2024-06-03T01:05:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.949+0000 7fa5f2a00700 1 -- 172.21.15.94:0/3463631951 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40173f0 con 0x7fa5fc104f70 2024-06-03T01:05:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.949+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc1a78d0 con 0x7fa5fc104f70 2024-06-03T01:05:14.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.951+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5fc069180 con 0x7fa5fc104f70 2024-06-03T01:05:14.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.954+0000 7fa5f2a00700 1 -- 172.21.15.94:0/3463631951 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5e4022470 con 0x7fa5fc104f70 2024-06-03T01:05:14.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.955+0000 7fa5f2a00700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa5e00a2ca0 0x7fa5e00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:14.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.955+0000 7fa5f2a00700 1 -- 172.21.15.94:0/3463631951 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7fa5e40cfe70 con 0x7fa5fc104f70 2024-06-03T01:05:14.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.956+0000 7fa5f9e00700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa5e00a2ca0 0x7fa5e00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:14.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.956+0000 7fa5f9e00700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa5e00a2ca0 0x7fa5e00a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009fd0 tx=0x7fa5ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:14.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:14.959+0000 7fa5f2a00700 1 -- 172.21.15.94:0/3463631951 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5e4013070 con 0x7fa5fc104f70 2024-06-03T01:05:15.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.236+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fa5fc02dca0 con 0x7fa5fc104f70 2024-06-03T01:05:15.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.324+0000 7fa5f2a00700 1 -- 172.21.15.94:0/3463631951 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7fa5e409cc70 con 0x7fa5fc104f70 2024-06-03T01:05:15.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.327+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa5e00a2ca0 msgr2=0x7fa5e00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:15.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.327+0000 7fa5fb200700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa5e00a2ca0 0x7fa5e00a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009fd0 tx=0x7fa5ec009260 comp rx=0 tx=0).stop 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.327+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc104f70 msgr2=0x7fa5fc19b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.327+0000 7fa5fb200700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc104f70 0x7fa5fc19b290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4005910 tx=0x7fa5e400fa00 comp rx=0 tx=0).stop 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.328+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 shutdown_connections 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.328+0000 7fa5fb200700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5fc10a660 0x7fa5fc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.328+0000 7fa5fb200700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa5e00a2ca0 0x7fa5e00a5160 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.328+0000 7fa5fb200700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5fc1045b0 0x7fa5fc19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.328+0000 7fa5fb200700 1 --2- 172.21.15.94:0/3463631951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5fc104f70 0x7fa5fc19b290 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:15.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.328+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 >> 172.21.15.94:0/3463631951 conn(0x7fa5fc1000c0 msgr2=0x7fa5fc101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:15.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.329+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 shutdown_connections 2024-06-03T01:05:15.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:15.329+0000 7fa5fb200700 1 -- 172.21.15.94:0/3463631951 wait complete. 2024-06-03T01:05:15.331 INFO:teuthology.orchestra.run.smithi094.stderr:added data pool 4 to fsmap 2024-06-03T01:05:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:15 smithi094 ceph-mon[32119]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:15 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3463631951' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T01:05:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:15 smithi179 ceph-mon[37041]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:15 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3463631951' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T01:05:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:15 smithi107 ceph-mon[36622]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:15 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3463631951' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-03T01:05:16.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:05:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:16 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3463631951' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T01:05:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:16 smithi094 ceph-mon[32119]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T01:05:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:16 smithi094 ceph-mon[32119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:05:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:16 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3463631951' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T01:05:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:16 smithi179 ceph-mon[37041]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T01:05:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:16 smithi179 ceph-mon[37041]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:05:16.608 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:16 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3463631951' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-03T01:05:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:16 smithi107 ceph-mon[36622]: osdmap e96: 12 total, 12 up, 12 in 2024-06-03T01:05:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:16 smithi107 ceph-mon[36622]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:05:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:17 smithi094 ceph-mon[32119]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:17 smithi094 ceph-mon[32119]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T01:05:17.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:17 smithi094 ceph-mon[32119]: Cluster is now healthy 2024-06-03T01:05:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:17 smithi179 ceph-mon[37041]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:17 smithi179 ceph-mon[37041]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T01:05:17.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:17 smithi179 ceph-mon[37041]: Cluster is now healthy 2024-06-03T01:05:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:17 smithi107 ceph-mon[36622]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:17 smithi107 ceph-mon[36622]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-03T01:05:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:17 smithi107 ceph-mon[36622]: Cluster is now healthy 2024-06-03T01:05:18.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 -- 172.21.15.94:0/2765855611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc1045c0 msgr2=0x7f10cc104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 --2- 172.21.15.94:0/2765855611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc1045c0 0x7f10cc104a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009990 tx=0x7f10b8009ca0 comp rx=0 tx=0).stop 2024-06-03T01:05:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 -- 172.21.15.94:0/2765855611 shutdown_connections 2024-06-03T01:05:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 --2- 172.21.15.94:0/2765855611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc10a650 0x7f10cc10aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 --2- 172.21.15.94:0/2765855611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10cc104f80 0x7f10cc111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 --2- 172.21.15.94:0/2765855611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc1045c0 0x7f10cc104a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 -- 172.21.15.94:0/2765855611 >> 172.21.15.94:0/2765855611 conn(0x7f10cc1000b0 msgr2=0x7f10cc1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:18.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 -- 172.21.15.94:0/2765855611 shutdown_connections 2024-06-03T01:05:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.865+0000 7f10d3000700 1 -- 172.21.15.94:0/2765855611 wait complete. 2024-06-03T01:05:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.866+0000 7f10d3000700 1 Processor -- start 2024-06-03T01:05:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.866+0000 7f10d3000700 1 -- start start 2024-06-03T01:05:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d3000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc1045c0 0x7f10cc1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d3000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc104f80 0x7f10cc1a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d3000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10cc10a650 0x7f10cc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d3000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc114960 con 0x7f10cc104f80 2024-06-03T01:05:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d3000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1147e0 con 0x7f10cc1045c0 2024-06-03T01:05:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d3000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc114ae0 con 0x7f10cc10a650 2024-06-03T01:05:18.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d1200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc104f80 0x7f10cc1a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc1045c0 0x7f10cc1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d1200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc104f80 0x7f10cc1a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33562/0 (socket says 172.21.15.94:33562) 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d1200700 1 -- 172.21.15.94:0/1204660070 learned_addr learned my addr 172.21.15.94:0/1204660070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.867+0000 7f10d2600700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10cc10a650 0x7f10cc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d1c00700 1 -- 172.21.15.94:0/1204660070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10cc10a650 msgr2=0x7f10cc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d1c00700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10cc10a650 0x7f10cc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d1c00700 1 -- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc104f80 msgr2=0x7f10cc1a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d1c00700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc104f80 0x7f10cc1a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d1c00700 1 -- 172.21.15.94:0/1204660070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc19b260 con 0x7f10cc1045c0 2024-06-03T01:05:18.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d1c00700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc1045c0 0x7f10cc1a0940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e9c0 tx=0x7f10c000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10ca000700 1 -- 172.21.15.94:0/1204660070 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c000cbd0 con 0x7f10cc1045c0 2024-06-03T01:05:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc19b550 con 0x7f10cc1045c0 2024-06-03T01:05:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10ca000700 1 -- 172.21.15.94:0/1204660070 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c0004500 con 0x7f10cc1045c0 2024-06-03T01:05:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10ca000700 1 -- 172.21.15.94:0/1204660070 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0003df0 con 0x7f10cc1045c0 2024-06-03T01:05:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.868+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a7810 con 0x7f10cc1045c0 2024-06-03T01:05:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.869+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc069180 con 0x7f10cc1045c0 2024-06-03T01:05:18.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.872+0000 7f10ca000700 1 -- 172.21.15.94:0/1204660070 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f10c0004000 con 0x7f10cc1045c0 2024-06-03T01:05:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.873+0000 7f10ca000700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10b40a2ca0 0x7f10b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.873+0000 7f10ca000700 1 -- 172.21.15.94:0/1204660070 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f10c00cf660 con 0x7f10cc1045c0 2024-06-03T01:05:18.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.873+0000 7f10d1200700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10b40a2ca0 0x7f10b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:18.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.874+0000 7f10d1200700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10b40a2ca0 0x7f10b40a5160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009750 tx=0x7f10b800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:18.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:18.875+0000 7f10ca000700 1 -- 172.21.15.94:0/1204660070 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10c001f340 con 0x7f10cc1045c0 2024-06-03T01:05:19.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.147+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10cc19bff0 con 0x7f10cc1045c0 2024-06-03T01:05:19.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.148+0000 7f10ca000700 1 -- 172.21.15.94:0/1204660070 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7f10c009c320 con 0x7f10cc1045c0 2024-06-03T01:05:19.148 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:05:19.149 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:05:15.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-06-03T01:05:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10b40a2ca0 msgr2=0x7f10b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10b40a2ca0 0x7f10b40a5160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009750 tx=0x7f10b800b280 comp rx=0 tx=0).stop 2024-06-03T01:05:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc1045c0 msgr2=0x7f10cc1a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:19.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc1045c0 0x7f10cc1a0940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e9c0 tx=0x7f10c000ecd0 comp rx=0 tx=0).stop 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 shutdown_connections 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10cc1045c0 0x7f10cc1a0940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10b40a2ca0 0x7f10b40a5160 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10cc10a650 0x7f10cc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 --2- 172.21.15.94:0/1204660070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10cc104f80 0x7f10cc1a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 >> 172.21.15.94:0/1204660070 conn(0x7f10cc1000b0 msgr2=0x7f10cc101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 shutdown_connections 2024-06-03T01:05:19.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:19.151+0000 7f10d3000700 1 -- 172.21.15.94:0/1204660070 wait complete. 2024-06-03T01:05:19.153 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 16 2024-06-03T01:05:19.583 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:19 smithi094 ceph-mon[32119]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:19.584 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:19 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1204660070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:05:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:19 smithi179 ceph-mon[37041]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:19 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1204660070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:05:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:19 smithi107 ceph-mon[36622]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:19.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:19 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1204660070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:05:20.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:05:20.488 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:20 smithi179 ceph-mon[37041]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:20.640 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:20 smithi094 ceph-mon[32119]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:20.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:20 smithi107 ceph-mon[36622]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:22.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 -- 172.21.15.94:0/478010095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 msgr2=0x7f11141088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:22.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 --2- 172.21.15.94:0/478010095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f11141088d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1108009990 tx=0x7f1108009ca0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 -- 172.21.15.94:0/478010095 shutdown_connections 2024-06-03T01:05:22.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 --2- 172.21.15.94:0/478010095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 0x7f111410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 --2- 172.21.15.94:0/478010095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f11141088d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 --2- 172.21.15.94:0/478010095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 0x7f1114102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 -- 172.21.15.94:0/478010095 >> 172.21.15.94:0/478010095 conn(0x7f1114079530 msgr2=0x7f1114079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:22.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 -- 172.21.15.94:0/478010095 shutdown_connections 2024-06-03T01:05:22.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.748+0000 7f111c400700 1 -- 172.21.15.94:0/478010095 wait complete. 2024-06-03T01:05:22.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.749+0000 7f111c400700 1 Processor -- start 2024-06-03T01:05:22.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.749+0000 7f111c400700 1 -- start start 2024-06-03T01:05:22.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111c400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 0x7f1114074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:22.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111c400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 0x7f1114074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:22.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111c400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f1114075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111c400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1114112780 con 0x7f11141084f0 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111c400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1114112600 con 0x7f1114102e80 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111c400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1114112900 con 0x7f11141024e0 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 0x7f1114074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111a400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f1114075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 0x7f1114074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 0x7f1114074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47810/0 (socket says 172.21.15.94:47810) 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f111a400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f1114075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33566/0 (socket says 172.21.15.94:33566) 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119000700 1 -- 172.21.15.94:0/2767892155 learned_addr learned my addr 172.21.15.94:0/2767892155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:22.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 0x7f1114074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:42760/0 (socket says 172.21.15.94:42760) 2024-06-03T01:05:22.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119a00700 1 -- 172.21.15.94:0/2767892155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 msgr2=0x7f1114074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:22.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119a00700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 0x7f1114074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119a00700 1 -- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 msgr2=0x7f1114075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119a00700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f1114075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.750+0000 7f1119a00700 1 -- 172.21.15.94:0/2767892155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11141a71c0 con 0x7f11141024e0 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f1119000700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 0x7f1114074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f111a400700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f1114075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f1119a00700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 0x7f1114074790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1108006010 tx=0x7f110800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f1112000700 1 -- 172.21.15.94:0/2767892155 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108027070 con 0x7f11141024e0 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11141a7450 con 0x7f11141024e0 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f1112000700 1 -- 172.21.15.94:0/2767892155 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1108003d70 con 0x7f11141024e0 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f1112000700 1 -- 172.21.15.94:0/2767892155 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110800f460 con 0x7f11141024e0 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.751+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11141a7900 con 0x7f11141024e0 2024-06-03T01:05:22.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.752+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1114069180 con 0x7f11141024e0 2024-06-03T01:05:22.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.754+0000 7f1112000700 1 -- 172.21.15.94:0/2767892155 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f110802a070 con 0x7f11141024e0 2024-06-03T01:05:22.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.755+0000 7f1112000700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10fc0a2c20 0x7f10fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:22.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.756+0000 7f1112000700 1 -- 172.21.15.94:0/2767892155 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f11080d0250 con 0x7f11141024e0 2024-06-03T01:05:22.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.756+0000 7f1119000700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10fc0a2c20 0x7f10fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:22.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.756+0000 7f1119000700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10fc0a2c20 0x7f10fc0a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1114079ba0 tx=0x7f1100009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:22.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.760+0000 7f1112000700 1 -- 172.21.15.94:0/2767892155 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f110805e850 con 0x7f11141024e0 2024-06-03T01:05:22.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.982+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f111402dca0 con 0x7f11141024e0 2024-06-03T01:05:22.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.984+0000 7f1112000700 1 -- 172.21.15.94:0/2767892155 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24127 (secure 0 0 0) 0x7f1108013070 con 0x7f11141024e0 2024-06-03T01:05:22.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:05:22.985 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":96,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:05:15.238154+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:05:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10fc0a2c20 msgr2=0x7f10fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10fc0a2c20 0x7f10fc0a50e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1114079ba0 tx=0x7f1100009260 comp rx=0 tx=0).stop 2024-06-03T01:05:22.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 msgr2=0x7f1114074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 0x7f1114074790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1108006010 tx=0x7f110800bab0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 shutdown_connections 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f10fc0a2c20 0x7f10fc0a50e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11141084f0 0x7f1114075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141024e0 0x7f1114074790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 --2- 172.21.15.94:0/2767892155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1114102e80 0x7f1114074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 >> 172.21.15.94:0/2767892155 conn(0x7f1114079530 msgr2=0x7f11141005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 shutdown_connections 2024-06-03T01:05:22.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:22.988+0000 7f111c400700 1 -- 172.21.15.94:0/2767892155 wait complete. 2024-06-03T01:05:23.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:23 smithi094 ceph-mon[32119]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:23.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:23 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2767892155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:23 smithi179 ceph-mon[37041]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:23.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:23 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2767892155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:23.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:23 smithi107 ceph-mon[36622]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:23.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:23 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2767892155' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:23.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:05:24.306 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:25.585 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:25 smithi107 ceph-mon[36622]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:25 smithi094 ceph-mon[32119]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:25.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:25 smithi179 ceph-mon[37041]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:26.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 -- 172.21.15.94:0/2382671901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 msgr2=0x7fd9cc1157c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:26.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/2382671901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 0x7fd9cc1157c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009f90 tx=0x7fd9c400b330 comp rx=0 tx=0).stop 2024-06-03T01:05:26.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 -- 172.21.15.94:0/2382671901 shutdown_connections 2024-06-03T01:05:26.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/2382671901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 0x7fd9cc1157c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/2382671901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9cc074830 0x7fd9cc074c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/2382671901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc075150 0x7fd9cc112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 -- 172.21.15.94:0/2382671901 >> 172.21.15.94:0/2382671901 conn(0x7fd9cc0fdf10 msgr2=0x7fd9cc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:26.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 -- 172.21.15.94:0/2382671901 shutdown_connections 2024-06-03T01:05:26.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.607+0000 7fd9cbe00700 1 -- 172.21.15.94:0/2382671901 wait complete. 2024-06-03T01:05:26.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 Processor -- start 2024-06-03T01:05:26.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 -- start start 2024-06-03T01:05:26.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc074830 0x7fd9cc19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:26.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9cc075150 0x7fd9cc19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:26.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 0x7fd9cc1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc118d10 con 0x7fd9cc074830 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc118b90 con 0x7fd9cc1133d0 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.609+0000 7fd9cbe00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc118e90 con 0x7fd9cc075150 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc074830 0x7fd9cc19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9ca000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9cc075150 0x7fd9cc19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc074830 0x7fd9cc19f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38528/0 (socket says 172.21.15.94:38528) 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 -- 172.21.15.94:0/3491810344 learned_addr learned my addr 172.21.15.94:0/3491810344 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9cb400700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 0x7fd9cc1a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 -- 172.21.15.94:0/3491810344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9cc075150 msgr2=0x7fd9cc19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9cc075150 0x7fd9cc19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 -- 172.21.15.94:0/3491810344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 msgr2=0x7fd9cc1a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 0x7fd9cc1a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.610+0000 7fd9caa00700 1 -- 172.21.15.94:0/3491810344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc1a4160 con 0x7fd9cc074830 2024-06-03T01:05:26.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.611+0000 7fd9caa00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc074830 0x7fd9cc19f1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00eb30 tx=0x7fd9bc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:26.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.611+0000 7fd9c3400700 1 -- 172.21.15.94:0/3491810344 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00cbd0 con 0x7fd9cc074830 2024-06-03T01:05:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.611+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc1a4450 con 0x7fd9cc074830 2024-06-03T01:05:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.611+0000 7fd9c3400700 1 -- 172.21.15.94:0/3491810344 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9bc004500 con 0x7fd9cc074830 2024-06-03T01:05:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.611+0000 7fd9c3400700 1 -- 172.21.15.94:0/3491810344 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc010430 con 0x7fd9cc074830 2024-06-03T01:05:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.611+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc1abcd0 con 0x7fd9cc074830 2024-06-03T01:05:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.612+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9cc069180 con 0x7fd9cc074830 2024-06-03T01:05:26.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.614+0000 7fd9c3400700 1 -- 172.21.15.94:0/3491810344 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd9bc003f00 con 0x7fd9cc074830 2024-06-03T01:05:26.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.616+0000 7fd9c3400700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd9b00a2c20 0x7fd9b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:26.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.616+0000 7fd9c3400700 1 -- 172.21.15.94:0/3491810344 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fd9bc0cf640 con 0x7fd9cc074830 2024-06-03T01:05:26.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.616+0000 7fd9ca000700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd9b00a2c20 0x7fd9b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.617+0000 7fd9ca000700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd9b00a2c20 0x7fd9b00a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009750 tx=0x7fd9b8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.621+0000 7fd9c3400700 1 -- 172.21.15.94:0/3491810344 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd9bc09c300 con 0x7fd9cc074830 2024-06-03T01:05:26.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.843+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9cc1abf60 con 0x7fd9cc074830 2024-06-03T01:05:26.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.848+0000 7fd9c3400700 1 -- 172.21.15.94:0/3491810344 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24127 (secure 0 0 0) 0x7fd9bc09bc00 con 0x7fd9cc074830 2024-06-03T01:05:26.849 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:05:26.849 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":96,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:05:15.238154+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.851+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd9b00a2c20 msgr2=0x7fd9b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.851+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd9b00a2c20 0x7fd9b00a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009750 tx=0x7fd9b8009310 comp rx=0 tx=0).stop 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.851+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc074830 msgr2=0x7fd9cc19f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.851+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc074830 0x7fd9cc19f1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00eb30 tx=0x7fd9bc00ee40 comp rx=0 tx=0).stop 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 shutdown_connections 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9cc1133d0 0x7fd9cc1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd9b00a2c20 0x7fd9b00a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9cc074830 0x7fd9cc19f1a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 --2- 172.21.15.94:0/3491810344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9cc075150 0x7fd9cc19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:26.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 >> 172.21.15.94:0/3491810344 conn(0x7fd9cc0fdf10 msgr2=0x7fd9cc0ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:26.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 shutdown_connections 2024-06-03T01:05:26.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:26.852+0000 7fd9cbe00700 1 -- 172.21.15.94:0/3491810344 wait complete. 2024-06-03T01:05:27.598 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:27 smithi094 ceph-mon[32119]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:27.598 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:27 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3491810344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:27 smithi179 ceph-mon[37041]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:27 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3491810344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:27.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:05:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:27 smithi107 ceph-mon[36622]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:27 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3491810344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:28.118 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:29.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:29 smithi094 ceph-mon[32119]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:29 smithi179 ceph-mon[37041]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:29 smithi107 ceph-mon[36622]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 -- 172.21.15.94:0/1545216077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec102540 msgr2=0x7f2bec1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/1545216077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec102540 0x7f2bec1029a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80098f0 tx=0x7f2bd8009c00 comp rx=0 tx=0).stop 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 -- 172.21.15.94:0/1545216077 shutdown_connections 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/1545216077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102ee0 0x7f2bec10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/1545216077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec107450 0x7f2bec107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/1545216077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec102540 0x7f2bec1029a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 -- 172.21.15.94:0/1545216077 >> 172.21.15.94:0/1545216077 conn(0x7f2bec0795f0 msgr2=0x7f2bec079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.439+0000 7f2bf0a00700 1 -- 172.21.15.94:0/1545216077 shutdown_connections 2024-06-03T01:05:30.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.440+0000 7f2bf0a00700 1 -- 172.21.15.94:0/1545216077 wait complete. 2024-06-03T01:05:30.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.441+0000 7f2bf0a00700 1 Processor -- start 2024-06-03T01:05:30.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.441+0000 7f2bf0a00700 1 -- start start 2024-06-03T01:05:30.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2bf0a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102540 0x7f2bec19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:30.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2bf0a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec102ee0 0x7f2bec19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:30.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2bf0a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec107450 0x7f2bec198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:30.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2bf0a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1127e0 con 0x7f2bec102ee0 2024-06-03T01:05:30.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2bf0a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec112660 con 0x7f2bec102540 2024-06-03T01:05:30.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2bf0a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec112960 con 0x7f2bec107450 2024-06-03T01:05:30.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2be9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec102ee0 0x7f2bec19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2be9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102540 0x7f2bec19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2be9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102540 0x7f2bec19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42556/0 (socket says 172.21.15.94:42556) 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2be9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec102ee0 0x7f2bec19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38544/0 (socket says 172.21.15.94:38544) 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2be9e00700 1 -- 172.21.15.94:0/930489677 learned_addr learned my addr 172.21.15.94:0/930489677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2bea800700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec107450 0x7f2bec198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.442+0000 7f2be9e00700 1 -- 172.21.15.94:0/930489677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec107450 msgr2=0x7f2bec198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be9e00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec107450 0x7f2bec198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be9e00700 1 -- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec102ee0 msgr2=0x7f2bec19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be9e00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec102ee0 0x7f2bec19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be9e00700 1 -- 172.21.15.94:0/930489677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec1991a0 con 0x7f2bec102540 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be9e00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102540 0x7f2bec19e880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9f0 tx=0x7f2bdc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be2a00700 1 -- 172.21.15.94:0/930489677 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004220 con 0x7f2bec102540 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be2a00700 1 -- 172.21.15.94:0/930489677 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc0043c0 con 0x7f2bec102540 2024-06-03T01:05:30.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec199490 con 0x7f2bec102540 2024-06-03T01:05:30.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2be2a00700 1 -- 172.21.15.94:0/930489677 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc003ce0 con 0x7f2bec102540 2024-06-03T01:05:30.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.443+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec0746c0 con 0x7f2bec102540 2024-06-03T01:05:30.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.444+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bec069180 con 0x7f2bec102540 2024-06-03T01:05:30.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.446+0000 7f2be2a00700 1 -- 172.21.15.94:0/930489677 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2bdc00c7c0 con 0x7f2bec102540 2024-06-03T01:05:30.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.447+0000 7f2be2a00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2bd00a2ca0 0x7f2bd00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:30.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.448+0000 7f2be2a00700 1 -- 172.21.15.94:0/930489677 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f2bdc0cf2b0 con 0x7f2bec102540 2024-06-03T01:05:30.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.448+0000 7f2be9400700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2bd00a2ca0 0x7f2bd00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:30.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.448+0000 7f2be9400700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2bd00a2ca0 0x7f2bd00a5160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80098c0 tx=0x7f2bd8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:30.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.452+0000 7f2be2a00700 1 -- 172.21.15.94:0/930489677 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2bdc09bfd0 con 0x7f2bec102540 2024-06-03T01:05:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:30 smithi094 ceph-mon[32119]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:30.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.725+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bec02dca0 con 0x7f2bec102540 2024-06-03T01:05:30.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.726+0000 7f2be2a00700 1 -- 172.21.15.94:0/930489677 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7f2bdc09b8d0 con 0x7f2bec102540 2024-06-03T01:05:30.726 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:05:30.727 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14388,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:05:15.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2bd00a2ca0 msgr2=0x7f2bd00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2bd00a2ca0 0x7f2bd00a5160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80098c0 tx=0x7f2bd8005dc0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102540 msgr2=0x7f2bec19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102540 0x7f2bec19e880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9f0 tx=0x7f2bdc00ed00 comp rx=0 tx=0).stop 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 shutdown_connections 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bec102540 0x7f2bec19e880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2bd00a2ca0 0x7f2bd00a5160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bec107450 0x7f2bec198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 --2- 172.21.15.94:0/930489677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bec102ee0 0x7f2bec19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.729+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 >> 172.21.15.94:0/930489677 conn(0x7f2bec0795f0 msgr2=0x7f2bec100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.730+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 shutdown_connections 2024-06-03T01:05:30.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:30.730+0000 7f2bf0a00700 1 -- 172.21.15.94:0/930489677 wait complete. 2024-06-03T01:05:30.731 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 16 2024-06-03T01:05:30.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:30 smithi107 ceph-mon[36622]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:30 smithi179 ceph-mon[37041]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:31.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:05:31.687 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:31 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/930489677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:05:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:31 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/930489677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:05:31.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:31 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/930489677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:05:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:32.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:32 smithi094 ceph-mon[32119]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:32.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:32 smithi107 ceph-mon[36622]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:32.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:32 smithi179 ceph-mon[37041]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:34.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.357+0000 7fbc9f800700 1 -- 172.21.15.94:0/429598221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc9810eab0 msgr2=0x7fbc9810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.357+0000 7fbc9f800700 1 --2- 172.21.15.94:0/429598221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc9810eab0 0x7fbc9810ee90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009990 tx=0x7fbc8c009ca0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.358+0000 7fbc9f800700 1 -- 172.21.15.94:0/429598221 shutdown_connections 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.358+0000 7fbc9f800700 1 --2- 172.21.15.94:0/429598221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc981094c0 0x7fbc98115980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.358+0000 7fbc9f800700 1 --2- 172.21.15.94:0/429598221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc9810eab0 0x7fbc9810ee90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.358+0000 7fbc9f800700 1 --2- 172.21.15.94:0/429598221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc98108b00 0x7fbc98108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.358+0000 7fbc9f800700 1 -- 172.21.15.94:0/429598221 >> 172.21.15.94:0/429598221 conn(0x7fbc98075c60 msgr2=0x7fbc98078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.358+0000 7fbc9f800700 1 -- 172.21.15.94:0/429598221 shutdown_connections 2024-06-03T01:05:34.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.358+0000 7fbc9f800700 1 -- 172.21.15.94:0/429598221 wait complete. 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.359+0000 7fbc9f800700 1 Processor -- start 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.359+0000 7fbc9f800700 1 -- start start 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.359+0000 7fbc9f800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc98108b00 0x7fbc9819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.359+0000 7fbc9f800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc981094c0 0x7fbc9819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9f800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc9810eab0 0x7fbc981a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9f800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98118de0 con 0x7fbc98108b00 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9f800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98118c60 con 0x7fbc9810eab0 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9f800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbc98118f60 con 0x7fbc981094c0 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc98108b00 0x7fbc9819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc98108b00 0x7fbc9819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38570/0 (socket says 172.21.15.94:38570) 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 -- 172.21.15.94:0/3038885812 learned_addr learned my addr 172.21.15.94:0/3038885812 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:34.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc97e00700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc981094c0 0x7fbc9819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:34.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9d800700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc9810eab0 0x7fbc981a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:34.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 -- 172.21.15.94:0/3038885812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc981094c0 msgr2=0x7fbc9819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:34.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc981094c0 0x7fbc9819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 -- 172.21.15.94:0/3038885812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc9810eab0 msgr2=0x7fbc981a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:34.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc9810eab0 0x7fbc981a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.360+0000 7fbc9ce00700 1 -- 172.21.15.94:0/3038885812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc981a42c0 con 0x7fbc98108b00 2024-06-03T01:05:34.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.361+0000 7fbc9ce00700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc98108b00 0x7fbc9819f2b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00bae0 tx=0x7fbc8c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.361+0000 7fbc95600700 1 -- 172.21.15.94:0/3038885812 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c026070 con 0x7fbc98108b00 2024-06-03T01:05:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.361+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc981a4550 con 0x7fbc98108b00 2024-06-03T01:05:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.361+0000 7fbc95600700 1 -- 172.21.15.94:0/3038885812 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c003d70 con 0x7fbc98108b00 2024-06-03T01:05:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.361+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc981abe30 con 0x7fbc98108b00 2024-06-03T01:05:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.361+0000 7fbc95600700 1 -- 172.21.15.94:0/3038885812 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c030060 con 0x7fbc98108b00 2024-06-03T01:05:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.362+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc98069180 con 0x7fbc98108b00 2024-06-03T01:05:34.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.365+0000 7fbc95600700 1 -- 172.21.15.94:0/3038885812 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbc8c029070 con 0x7fbc98108b00 2024-06-03T01:05:34.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.366+0000 7fbc95600700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc800a2c20 0x7fbc800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:34.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.366+0000 7fbc95600700 1 -- 172.21.15.94:0/3038885812 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7fbc8c0d08d0 con 0x7fbc98108b00 2024-06-03T01:05:34.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.367+0000 7fbc97e00700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc800a2c20 0x7fbc800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:34.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.367+0000 7fbc97e00700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc800a2c20 0x7fbc800a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009680 tx=0x7fbc84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:34.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.371+0000 7fbc95600700 1 -- 172.21.15.94:0/3038885812 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc8c013070 con 0x7fbc98108b00 2024-06-03T01:05:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-03T01:05:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-03T01:05:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-06-03T01:05:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-06-03T01:05:34.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-06-03T01:05:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-06-03T01:05:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-06-03T01:05:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-06-03T01:05:34.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:34 smithi107 ceph-mon[36622]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-06-03T01:05:34.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-06-03T01:05:34.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:34 smithi094 ceph-mon[32119]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T01:05:34.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.602+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc9802dca0 con 0x7fbc98108b00 2024-06-03T01:05:34.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.604+0000 7fbc95600700 1 -- 172.21.15.94:0/3038885812 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+24655 (secure 0 0 0) 0x7fbc8c09d500 con 0x7fbc98108b00 2024-06-03T01:05:34.605 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:05:34.605 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":97,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:05:34.236452+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-06-03T01:05:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-06-03T01:05:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:34 smithi179 ceph-mon[37041]: osdmap e97: 12 total, 12 up, 12 in 2024-06-03T01:05:34.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc800a2c20 msgr2=0x7fbc800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:34.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc800a2c20 0x7fbc800a50e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009680 tx=0x7fbc84009260 comp rx=0 tx=0).stop 2024-06-03T01:05:34.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc98108b00 msgr2=0x7fbc9819f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:34.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc98108b00 0x7fbc9819f2b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00bae0 tx=0x7fbc8c00bb10 comp rx=0 tx=0).stop 2024-06-03T01:05:34.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 shutdown_connections 2024-06-03T01:05:34.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc9810eab0 0x7fbc981a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbc800a2c20 0x7fbc800a50e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc98108b00 0x7fbc9819f2b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 --2- 172.21.15.94:0/3038885812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc981094c0 0x7fbc9819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:34.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.607+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 >> 172.21.15.94:0/3038885812 conn(0x7fbc98075c60 msgr2=0x7fbc98076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:34.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.608+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 shutdown_connections 2024-06-03T01:05:34.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:34.608+0000 7fbc9f800700 1 -- 172.21.15.94:0/3038885812 wait complete. 2024-06-03T01:05:35.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-03T01:04:57.629349+0000', 'modified': '2024-06-03T01:05:15.238340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.107:6839/1423733861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1423733861}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1423733861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24524, 'qdb_cluster': [24524]} 2024-06-03T01:05:35.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-03T01:05:35.428 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T01:05:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:35 smithi107 ceph-mon[36622]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:35 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3038885812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:35 smithi107 ceph-mon[36622]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T01:05:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:35 smithi094 ceph-mon[32119]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:35 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3038885812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:35 smithi094 ceph-mon[32119]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T01:05:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:35 smithi179 ceph-mon[37041]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:35 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3038885812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:05:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:35 smithi179 ceph-mon[37041]: osdmap e98: 12 total, 12 up, 12 in 2024-06-03T01:05:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:37 smithi094 ceph-mon[32119]: pgmap v359: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:37 smithi179 ceph-mon[37041]: pgmap v359: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:37.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:37 smithi107 ceph-mon[36622]: pgmap v359: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:38.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.181+0000 7fce64a00700 1 -- 172.21.15.94:0/3071265959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60102740 msgr2=0x7fce60102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.181+0000 7fce64a00700 1 --2- 172.21.15.94:0/3071265959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60102740 0x7fce60102b20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce500099c0 tx=0x7fce50009cd0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.182+0000 7fce64a00700 1 -- 172.21.15.94:0/3071265959 shutdown_connections 2024-06-03T01:05:38.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.182+0000 7fce64a00700 1 --2- 172.21.15.94:0/3071265959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce60107840 0x7fce6010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.182+0000 7fce64a00700 1 --2- 172.21.15.94:0/3071265959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60102740 0x7fce60102b20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.182+0000 7fce64a00700 1 --2- 172.21.15.94:0/3071265959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce601030f0 0x7fce60107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.182+0000 7fce64a00700 1 -- 172.21.15.94:0/3071265959 >> 172.21.15.94:0/3071265959 conn(0x7fce600795f0 msgr2=0x7fce60079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:38.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.182+0000 7fce64a00700 1 -- 172.21.15.94:0/3071265959 shutdown_connections 2024-06-03T01:05:38.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.182+0000 7fce64a00700 1 -- 172.21.15.94:0/3071265959 wait complete. 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 Processor -- start 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 -- start start 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce60102740 0x7fce6019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce601030f0 0x7fce6019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60107840 0x7fce6019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce601127b0 con 0x7fce60107840 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fce60112630 con 0x7fce60102740 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.183+0000 7fce64a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fce60112930 con 0x7fce601030f0 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60107840 0x7fce6019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce60102740 0x7fce6019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce55400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce601030f0 0x7fce6019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60107840 0x7fce6019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55448/0 (socket says 172.21.15.94:55448) 2024-06-03T01:05:38.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5e800700 1 -- 172.21.15.94:0/349509142 learned_addr learned my addr 172.21.15.94:0/349509142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:38.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5de00700 1 -- 172.21.15.94:0/349509142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce601030f0 msgr2=0x7fce6019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5de00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce601030f0 0x7fce6019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5de00700 1 -- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60107840 msgr2=0x7fce6019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5de00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60107840 0x7fce6019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5de00700 1 -- 172.21.15.94:0/349509142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6019fe00 con 0x7fce60102740 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.184+0000 7fce5e800700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60107840 0x7fce6019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.185+0000 7fce5de00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce60102740 0x7fce6019adf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce500098e0 tx=0x7fce50003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.185+0000 7fce57400700 1 -- 172.21.15.94:0/349509142 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50003680 con 0x7fce60102740 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.185+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce601a0090 con 0x7fce60102740 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.185+0000 7fce57400700 1 -- 172.21.15.94:0/349509142 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce500055a0 con 0x7fce60102740 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.185+0000 7fce57400700 1 -- 172.21.15.94:0/349509142 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5000fc90 con 0x7fce60102740 2024-06-03T01:05:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.185+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce601a7970 con 0x7fce60102740 2024-06-03T01:05:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.190+0000 7fce57400700 1 -- 172.21.15.94:0/349509142 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fce5000f460 con 0x7fce60102740 2024-06-03T01:05:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 Processor -- start 2024-06-03T01:05:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 -- start start 2024-06-03T01:05:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce57400700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce440a2dc0 0x7fce440a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43ed6df0 0x55be43f5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f5a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be43e699c0 con 0x55be43ed6df0 2024-06-03T01:05:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be43eacab0 con 0x55be43f5a560 2024-06-03T01:05:38.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce6f475b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55be43e678a0 con 0x55be43f652b0 2024-06-03T01:05:38.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.191+0000 7fce57400700 1 -- 172.21.15.94:0/349509142 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fce500d01e0 con 0x7fce60102740 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43ed6df0 0x55be43f5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce55400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f5a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5e800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce55400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f5a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57962/0 (socket says 172.21.15.94:57962) 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce55400700 1 -- 172.21.15.94:0/610277648 learned_addr learned my addr 172.21.15.94:0/610277648 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5e800700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce440a2dc0 0x7fce440a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5de00700 1 -- 172.21.15.94:0/610277648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 msgr2=0x55be43f676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5de00700 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f676a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5de00700 1 -- 172.21.15.94:0/610277648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 msgr2=0x55be43f5a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5de00700 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f5a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5de00700 1 -- 172.21.15.94:0/610277648 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be43e54f40 con 0x55be43ed6df0 2024-06-03T01:05:38.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce55400700 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f5a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:38.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.192+0000 7fce5de00700 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43ed6df0 0x55be43f5a020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce5009c410 tx=0x7fce50003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ab4077637e760f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:38.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.193+0000 7fce5e800700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce440a2dc0 0x7fce440a5280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce58009fd0 tx=0x7fce5800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:38.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.193+0000 7fce4f400700 1 -- 172.21.15.94:0/610277648 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce500095c0 con 0x55be43ed6df0 2024-06-03T01:05:38.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.193+0000 7fce4f400700 1 -- 172.21.15.94:0/610277648 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5000fe30 con 0x55be43ed6df0 2024-06-03T01:05:38.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.193+0000 7fce4f400700 1 -- 172.21.15.94:0/610277648 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5009aa10 con 0x55be43ed6df0 2024-06-03T01:05:38.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.193+0000 7fce6f475b80 1 -- 172.21.15.94:0/610277648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43ed6df0 msgr2=0x55be43f5a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.193+0000 7fce6f475b80 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43ed6df0 0x55be43f5a020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce5009c410 tx=0x7fce50003f60 comp rx=0 tx=0).stop 2024-06-03T01:05:38.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 -- 172.21.15.94:0/610277648 shutdown_connections 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43ed6df0 0x55be43f5a020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 --2- 172.21.15.94:0/610277648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f5a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 -- 172.21.15.94:0/610277648 >> 172.21.15.94:0/610277648 conn(0x55be43e87120 msgr2=0x55be43f56460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 -- 172.21.15.94:0/610277648 shutdown_connections 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 -- 172.21.15.94:0/610277648 wait complete. 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.194+0000 7fce6f475b80 1 -- start start 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 1 Processor -- start 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f64a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f61d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f62260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be43e48f10 con 0x55be43f652b0 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be43ea8a70 con 0x55be43f5a560 2024-06-03T01:05:38.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55be43f67d20 con 0x55be43ed6df0 2024-06-03T01:05:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce6f475b80 10 client.0 _ll_register_callbacks cb 0x55be43e466d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:05:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce5e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f62260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce5de00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f64a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce55400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f61d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce5de00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f64a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43628/0 (socket says 172.21.15.94:43628) 2024-06-03T01:05:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce55400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f61d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57976/0 (socket says 172.21.15.94:57976) 2024-06-03T01:05:38.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.195+0000 7fce5de00700 1 -- 172.21.15.94:0/2865293878 learned_addr learned my addr 172.21.15.94:0/2865293878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce55400700 1 -- 172.21.15.94:0/2865293878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 msgr2=0x55be43f64a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce55400700 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f64a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce55400700 1 -- 172.21.15.94:0/2865293878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 msgr2=0x55be43f62260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce55400700 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f62260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce55400700 1 -- 172.21.15.94:0/2865293878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be43dfb880 con 0x55be43f5a560 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce55400700 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f61d20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce480095f0 tx=0x7fce48011a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaab3e64767032ce server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce43e00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce43e00700 1 -- 172.21.15.94:0/2865293878 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48004320 con 0x55be43f5a560 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5000fe30 con 0x55be43f5a560 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.196+0000 7fce43e00700 1 -- 172.21.15.94:0/2865293878 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fce480044c0 con 0x55be43f5a560 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.197+0000 7fce43e00700 1 -- 172.21.15.94:0/2865293878 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fce48025050 con 0x55be43f5a560 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.197+0000 7fce6f475b80 10 client.24620 fetch_fsmap learned FSMap version 16 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.197+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55be43f72420 con 0x55be43f5a560 2024-06-03T01:05:38.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.197+0000 7fce43e00700 1 -- 172.21.15.94:0/2865293878 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5000fe30 con 0x55be43f5a560 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.197+0000 7fce43e00700 1 -- 172.21.15.94:0/2865293878 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7fce48003680 con 0x55be43f5a560 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.197+0000 7fce6f475b80 10 client.24620 fetch_fsmap finished waiting for FSMap version 16 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce6f475b80 10 client.24620 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce6f475b80 20 client.24620 populate_metadata read hostname 'smithi094' 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43f74a60 0x55be43f75ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce6f475b80 4 client.24620 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be43eb9fb0 con 0x55be43f74a60 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce5de00700 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43f74a60 0x55be43f75ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce5de00700 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43f74a60 0x55be43f75ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146b9ec7f95c9469 server_cookie=f11a2496924b6073 in_seq=0 out_seq=0 2024-06-03T01:05:38.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.198+0000 7fce43e00700 10 client.24620 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:05:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.199+0000 7fce43e00700 1 -- 172.21.15.94:0/2865293878 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be43eb9fb0 con 0x55be43f74a60 2024-06-03T01:05:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.199+0000 7fce43e00700 10 client.24620 handle_command_reply: tid=0 2024-06-03T01:05:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.199+0000 7fce6f475b80 1 client.24620 shutdown 2024-06-03T01:05:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 msgr2=0x55be43f61d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f61d20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce480095f0 tx=0x7fce48011a10 comp rx=0 tx=0).stop 2024-06-03T01:05:38.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 shutdown_connections 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f62260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f64a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43f74a60 0x55be43f75ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2865293878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f61d20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 >> 172.21.15.94:0/2865293878 conn(0x55be43e87120 msgr2=0x55be43f556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce43e00700 0 client.24620 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 shutdown_connections 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 1 -- 172.21.15.94:0/2865293878 wait complete. 2024-06-03T01:05:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.200+0000 7fce6f475b80 20 client.24620 trim_cache size 0 max 16384 2024-06-03T01:05:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 Processor -- start 2024-06-03T01:05:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 -- start start 2024-06-03T01:05:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f565d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f55570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f64560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be43f74210 con 0x55be43f652b0 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be43f748e0 con 0x55be43f5a560 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.223+0000 7fce6f475b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55be43f74fb0 con 0x55be43ed6df0 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce5e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f64560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f55570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce5de00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f55570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57988/0 (socket says 172.21.15.94:57988) 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce5e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f64560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55470/0 (socket says 172.21.15.94:55470) 2024-06-03T01:05:38.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 -- 172.21.15.94:0/2926707074 learned_addr learned my addr 172.21.15.94:0/2926707074 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 -- 172.21.15.94:0/2926707074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 msgr2=0x55be43f565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 -- 172.21.15.94:0/2926707074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 msgr2=0x55be43f64560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f64560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 -- 172.21.15.94:0/2926707074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be43f5af80 con 0x55be43f5a560 2024-06-03T01:05:38.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce5de00700 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.224+0000 7fce55400700 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f55570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce48009340 tx=0x7fce480096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2bf5803155bbe68 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.225+0000 7fce4ea00700 1 -- 172.21.15.94:0/2926707074 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48000e60 con 0x55be43f5a560 2024-06-03T01:05:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.225+0000 7fce4ea00700 1 -- 172.21.15.94:0/2926707074 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48003dd0 con 0x55be43f5a560 2024-06-03T01:05:38.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.225+0000 7fce4ea00700 1 -- 172.21.15.94:0/2926707074 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48010910 con 0x55be43f5a560 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 -- 172.21.15.94:0/2926707074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 msgr2=0x55be43f55570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f55570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce48009340 tx=0x7fce480096d0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 -- 172.21.15.94:0/2926707074 shutdown_connections 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f652b0 0x55be43f64560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43ed6df0 0x55be43f565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2926707074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43f5a560 0x55be43f55570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 -- 172.21.15.94:0/2926707074 >> 172.21.15.94:0/2926707074 conn(0x55be43e87120 msgr2=0x55be43f54e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 -- 172.21.15.94:0/2926707074 shutdown_connections 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 -- 172.21.15.94:0/2926707074 wait complete. 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.226+0000 7fce6f475b80 1 -- start start 2024-06-03T01:05:38.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 1 Processor -- start 2024-06-03T01:05:38.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43ed6df0 0x55be43f5e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f5a560 0x55be43f5e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be43f75d50 con 0x55be43f5a560 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be43f75680 con 0x55be43ed6df0 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55be43f76420 con 0x55be43f652b0 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce6f475b80 10 client.0 _ll_register_callbacks cb 0x55be43e466d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce55400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f5a560 0x55be43f5e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce5de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43ed6df0 0x55be43f5e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce55400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f5a560 0x55be43f5e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55472/0 (socket says 172.21.15.94:55472) 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.227+0000 7fce55400700 1 -- 172.21.15.94:0/2333500874 learned_addr learned my addr 172.21.15.94:0/2333500874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce5e800700 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce5de00700 1 -- 172.21.15.94:0/2333500874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 msgr2=0x55be43f7e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce5de00700 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f7e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce5de00700 1 -- 172.21.15.94:0/2333500874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f5a560 msgr2=0x55be43f5e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:05:38.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce5de00700 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f5a560 0x55be43f5e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce5de00700 1 -- 172.21.15.94:0/2333500874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be43f7e880 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce5de00700 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43ed6df0 0x55be43f5e390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce500a4dc0 tx=0x7fce50003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1704d679a12684e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce43e00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce43e00700 1 -- 172.21.15.94:0/2333500874 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce500b9070 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.228+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce48003dd0 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce43e00700 1 -- 172.21.15.94:0/2333500874 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fce5009a7b0 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce43e00700 1 -- 172.21.15.94:0/2333500874 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fce5009c4e0 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 10 client.24626 fetch_fsmap learned FSMap version 16 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55be43f7fb80 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce43e00700 1 -- 172.21.15.94:0/2333500874 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48003dd0 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce43e00700 1 -- 172.21.15.94:0/2333500874 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7fce50020370 con 0x55be43ed6df0 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 10 client.24626 fetch_fsmap finished waiting for FSMap version 16 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 10 client.24626 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 20 client.24626 populate_metadata read hostname 'smithi094' 2024-06-03T01:05:38.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43fecea0 0x55be43febbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:38.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 4 client.24626 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:05:38.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.229+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "status"}) -- 0x55be43eb4530 con 0x55be43fecea0 2024-06-03T01:05:38.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce55400700 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43fecea0 0x55be43febbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce43e00700 10 client.24626 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce55400700 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43fecea0 0x55be43febbb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58cfd5a00690105 server_cookie=91736ca21bf6ad91 in_seq=0 out_seq=0 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce43e00700 1 -- 172.21.15.94:0/2333500874 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55be43eb4530 con 0x55be43fecea0 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce43e00700 10 client.24626 handle_command_reply: tid=0 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce6f475b80 1 client.24626 shutdown 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43ed6df0 msgr2=0x55be43f5e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.230+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43ed6df0 0x55be43f5e390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce500a4dc0 tx=0x7fce50003ed0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 shutdown_connections 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be43fecea0 0x55be43febbb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be43ed6df0 0x55be43f5e390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be43f652b0 0x55be43f7e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 --2- 172.21.15.94:0/2333500874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be43f5a560 0x55be43f5e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce43e00700 0 client.24626 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 >> 172.21.15.94:0/2333500874 conn(0x55be43e87120 msgr2=0x55be43f556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 shutdown_connections 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 1 -- 172.21.15.94:0/2333500874 wait complete. 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce6f475b80 20 client.24626 trim_cache size 0 max 16384 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24524, 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:05:38.234 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 40.547338134, 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 16, 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 98, 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 95, 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 64.633475770000004 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:05:38.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce440a2dc0 msgr2=0x7fce440a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce64a00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce440a2dc0 0x7fce440a5280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce58009fd0 tx=0x7fce5800b040 comp rx=0 tx=0).stop 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce60102740 msgr2=0x7fce6019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.231+0000 7fce64a00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce60102740 0x7fce6019adf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce500098e0 tx=0x7fce50003a90 comp rx=0 tx=0).stop 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 shutdown_connections 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce60102740 0x7fce6019adf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fce440a2dc0 0x7fce440a5280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce60107840 0x7fce6019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 --2- 172.21.15.94:0/349509142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce601030f0 0x7fce6019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 >> 172.21.15.94:0/349509142 conn(0x7fce600795f0 msgr2=0x7fce60101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 shutdown_connections 2024-06-03T01:05:38.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:38.232+0000 7fce64a00700 1 -- 172.21.15.94:0/349509142 wait complete. 2024-06-03T01:05:39.163 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T01:05:39.163 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:39.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T01:05:39.593 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:39.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:39 smithi094 ceph-mon[32119]: pgmap v360: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:39.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:39 smithi179 ceph-mon[37041]: pgmap v360: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:39.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:39 smithi107 ceph-mon[36622]: pgmap v360: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:41.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:41 smithi094 ceph-mon[32119]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:41.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:41 smithi179 ceph-mon[37041]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:41.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:41 smithi107 ceph-mon[36622]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:41.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.911+0000 7fdf06c00700 1 -- 172.21.15.94:0/3563643434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 msgr2=0x7fdf0010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:41.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.911+0000 7fdf06c00700 1 --2- 172.21.15.94:0/3563643434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 0x7fdf0010f330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdef800b0a0 tx=0x7fdef800b3b0 comp rx=0 tx=0).stop 2024-06-03T01:05:41.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.911+0000 7fdf06c00700 1 -- 172.21.15.94:0/3563643434 shutdown_connections 2024-06-03T01:05:41.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.911+0000 7fdf06c00700 1 --2- 172.21.15.94:0/3563643434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf001084f0 0x7fdf001088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:41.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.911+0000 7fdf06c00700 1 --2- 172.21.15.94:0/3563643434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 0x7fdf0010f330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:41.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.911+0000 7fdf06c00700 1 --2- 172.21.15.94:0/3563643434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf00102550 0x7fdf001029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:41.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.911+0000 7fdf06c00700 1 -- 172.21.15.94:0/3563643434 >> 172.21.15.94:0/3563643434 conn(0x7fdf00079650 msgr2=0x7fdf00079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:41.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.912+0000 7fdf06c00700 1 -- 172.21.15.94:0/3563643434 shutdown_connections 2024-06-03T01:05:41.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.912+0000 7fdf06c00700 1 -- 172.21.15.94:0/3563643434 wait complete. 2024-06-03T01:05:41.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 Processor -- start 2024-06-03T01:05:41.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 -- start start 2024-06-03T01:05:41.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf00102550 0x7fdf0019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:41.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 0x7fdf0019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf001084f0 0x7fdf00198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001127c0 con 0x7fdf001084f0 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00112640 con 0x7fdf00102550 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.913+0000 7fdf06c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00112940 con 0x7fdf00102f10 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf001084f0 0x7fdf00198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdeffe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf00102550 0x7fdf0019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf001084f0 0x7fdf00198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55486/0 (socket says 172.21.15.94:55486) 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdeffe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf00102550 0x7fdf0019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:58018/0 (socket says 172.21.15.94:58018) 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 -- 172.21.15.94:0/243445918 learned_addr learned my addr 172.21.15.94:0/243445918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdeff400700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 0x7fdf0019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 -- 172.21.15.94:0/243445918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 msgr2=0x7fdf0019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 0x7fdf0019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 -- 172.21.15.94:0/243445918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf00102550 msgr2=0x7fdf0019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:41.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf00102550 0x7fdf0019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:41.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdf04c00700 1 -- 172.21.15.94:0/243445918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf00199160 con 0x7fdf001084f0 2024-06-03T01:05:41.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.914+0000 7fdeffe00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf00102550 0x7fdf0019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:41.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.915+0000 7fdf04c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf001084f0 0x7fdf00198a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdef800b070 tx=0x7fdef8004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:41.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.915+0000 7fdefcc00700 1 -- 172.21.15.94:0/243445918 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8021070 con 0x7fdf001084f0 2024-06-03T01:05:41.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.915+0000 7fdefcc00700 1 -- 172.21.15.94:0/243445918 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef8003c00 con 0x7fdf001084f0 2024-06-03T01:05:41.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.915+0000 7fdefcc00700 1 -- 172.21.15.94:0/243445918 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8004740 con 0x7fdf001084f0 2024-06-03T01:05:41.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.915+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf001993f0 con 0x7fdf001084f0 2024-06-03T01:05:41.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.915+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf001a5710 con 0x7fdf001084f0 2024-06-03T01:05:41.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.916+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0010a9f0 con 0x7fdf001084f0 2024-06-03T01:05:41.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.918+0000 7fdefcc00700 1 -- 172.21.15.94:0/243445918 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdef8007980 con 0x7fdf001084f0 2024-06-03T01:05:41.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.919+0000 7fdefcc00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdee40a2ca0 0x7fdee40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:41.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.920+0000 7fdefcc00700 1 -- 172.21.15.94:0/243445918 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fdef800f020 con 0x7fdf001084f0 2024-06-03T01:05:41.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.920+0000 7fdeffe00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdee40a2ca0 0x7fdee40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:41.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.920+0000 7fdeffe00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdee40a2ca0 0x7fdee40a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdf00199b40 tx=0x7fdef0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:41.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:41.922+0000 7fdefcc00700 1 -- 172.21.15.94:0/243445918 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdef802a020 con 0x7fdf001084f0 2024-06-03T01:05:42.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.194+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdf00199f30 con 0x7fdf001084f0 2024-06-03T01:05:42.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.195+0000 7fdefcc00700 1 -- 172.21.15.94:0/243445918 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7fdef809d6d0 con 0x7fdf001084f0 2024-06-03T01:05:42.197 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-03T01:05:42.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.198+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdee40a2ca0 msgr2=0x7fdee40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:42.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.198+0000 7fdf06c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdee40a2ca0 0x7fdee40a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdf00199b40 tx=0x7fdef0009260 comp rx=0 tx=0).stop 2024-06-03T01:05:42.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.198+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf001084f0 msgr2=0x7fdf00198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:42.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.198+0000 7fdf06c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf001084f0 0x7fdf00198a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdef800b070 tx=0x7fdef8004c60 comp rx=0 tx=0).stop 2024-06-03T01:05:42.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 shutdown_connections 2024-06-03T01:05:42.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf00102550 0x7fdf0019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:42.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdee40a2ca0 0x7fdee40a5160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:42.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf001084f0 0x7fdf00198a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:42.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 --2- 172.21.15.94:0/243445918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf00102f10 0x7fdf0019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:42.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 >> 172.21.15.94:0/243445918 conn(0x7fdf00079650 msgr2=0x7fdf00100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:42.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 shutdown_connections 2024-06-03T01:05:42.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:42.199+0000 7fdf06c00700 1 -- 172.21.15.94:0/243445918 wait complete. 2024-06-03T01:05:42.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:42 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/243445918' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:05:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:42 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/243445918' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:05:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:42 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/243445918' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:05:43.173 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T01:05:43.174 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-03T01:05:43.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:43.174 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-06-03T01:05:43.221 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T01:05:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T01:05:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T01:05:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:05:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-06-03T01:05:43.221 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:05:43.221 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42013sec preferred_lft 42013sec 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-06-03T01:05:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:05:43.224 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T01:05:43.224 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:05:43.224 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:05:43.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-06-03T01:05:43.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-06-03T01:05:43.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-06-03T01:05:43.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T01:05:43.224 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:05:43.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:05:43.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:43 smithi094 ceph-mon[32119]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:43.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:43 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:05:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:43 smithi179 ceph-mon[37041]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:43.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:43 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:05:43.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:05:43.723 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:43 smithi107 ceph-mon[36622]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:43.723 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:43 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:05:43.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:43.726 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T01:05:43.792 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-06-03T01:05:43.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:43.796 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-06-03T01:05:43.847 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-06-03T01:05:43.847 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T01:05:43.847 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-03T01:05:43.847 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-03T01:05:43.847 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T01:05:43.848 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:05:43.848 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:05:43.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-03T01:05:43.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-03T01:05:43.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-03T01:05:43.848 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:05:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:05:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:44 smithi094 ceph-mon[32119]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:44.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:44 smithi107 ceph-mon[36622]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:44.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:44 smithi179 ceph-mon[37041]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:45.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:05:45.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:45.015 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-06-03T01:05:45.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:45.060 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-06-03T01:05:45.143 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:05:45.143 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:05:45.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:05:45.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-03T01:05:45.143 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:05:45.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:05:45.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:05:45.341 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-03T01:05:45.342 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:05:45.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:05:45.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:05:45.631 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:05:45.631 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:05:45.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-06-03T01:05:45.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T01:05:45.631 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:05:45.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:05:45.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:45.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:45.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:45.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:45 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:45.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:05:45.834 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-03T01:05:45.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:05:45.835 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:45.874 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-03T01:05:45.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:05:45.875 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:45.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-06-03T01:05:46.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:45 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:45 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:45 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:45 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.073 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T01:05:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:45 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:46.113 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-06-03T01:05:46.113 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-06-03T01:05:46.113 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-06-03T01:05:46.114 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-06-03T01:05:46.114 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-06-03T01:05:46.114 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-06-03T01:05:46.114 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-06-03T01:05:46.114 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-06-03T01:05:46.115 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-06-03T01:05:46.115 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-06-03T01:05:46.115 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-06-03T01:05:46.115 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-06-03T01:05:46.115 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T01:05:46.115 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-06-03T01:05:46.116 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-06-03T01:05:46.116 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-06-03T01:05:46.116 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-06-03T01:05:46.116 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-06-03T01:05:46.116 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-06-03T01:05:46.116 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T01:05:46.117 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T01:05:46.117 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T01:05:46.117 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T01:05:46.117 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-06-03T01:05:46.117 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-06-03T01:05:46.117 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay 2024-06-03T01:05:46.118 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/a0ab7bf4d975568007299b5077713b847111ebac4c5cb7a2fc755355d7f013f1/merged 2024-06-03T01:05:46.118 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/90d614b5bb1d918ec32f7a1f87529411cefb058a431b01759101d8a3525812a1/merged 2024-06-03T01:05:46.118 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/0 2024-06-03T01:05:46.118 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T01:05:46.118 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6f0ccb81e202b4cd654a1dc4b16d0cec710790e01594ef3ee41f92c1266e850b/merged 2024-06-03T01:05:46.119 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/8a52d6ca46f848655447f8a4616102989fe09d873b53c9fb3662ed3a82859a15/merged 2024-06-03T01:05:46.119 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/dae867281ca3ecfa954674c2c6daf8fe594b92c75e8540d97eeead8063c3bd6c/merged 2024-06-03T01:05:46.119 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6a8870a92b7ee7aabf7ad7f5f348508704c6aa4c6be28c278bc34ccacec87a13/merged 2024-06-03T01:05:46.119 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/2590eafe96576f9411553d7d7118b8529eb4290469d78f4417ed7b66283f2435/merged 2024-06-03T01:05:46.119 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/01d99acd279c0cc27a5dd3e648b5e8245db954ed3e78af855619df7bf09183a7/merged 2024-06-03T01:05:46.119 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/cd3d0366dc520e3c5eaf673db688e1891d7cb4639a7cbd0b320e1644c9d92b50/merged 2024-06-03T01:05:46.120 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/4d84f59a7a275a54d905f0784d44984b7a139202e0392f1ccfa170be620dd35e/merged 2024-06-03T01:05:46.120 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-06-03T01:05:46.120 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:05:46.120 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:05:46.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:46.122 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-06-03T01:05:46.215 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-03T01:05:46.215 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-03T01:05:46.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-06-03T01:05:46.287 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T01:05:46.326 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-06-03T01:05:46.326 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-06-03T01:05:46.326 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-06-03T01:05:46.326 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-06-03T01:05:46.326 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-06-03T01:05:46.326 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-06-03T01:05:46.327 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/a0ab7bf4d975568007299b5077713b847111ebac4c5cb7a2fc755355d7f013f1/merged 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/90d614b5bb1d918ec32f7a1f87529411cefb058a431b01759101d8a3525812a1/merged 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/0 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6f0ccb81e202b4cd654a1dc4b16d0cec710790e01594ef3ee41f92c1266e850b/merged 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/8a52d6ca46f848655447f8a4616102989fe09d873b53c9fb3662ed3a82859a15/merged 2024-06-03T01:05:46.328 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/dae867281ca3ecfa954674c2c6daf8fe594b92c75e8540d97eeead8063c3bd6c/merged 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6a8870a92b7ee7aabf7ad7f5f348508704c6aa4c6be28c278bc34ccacec87a13/merged 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/2590eafe96576f9411553d7d7118b8529eb4290469d78f4417ed7b66283f2435/merged 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/01d99acd279c0cc27a5dd3e648b5e8245db954ed3e78af855619df7bf09183a7/merged 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/cd3d0366dc520e3c5eaf673db688e1891d7cb4639a7cbd0b320e1644c9d92b50/merged 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/4d84f59a7a275a54d905f0784d44984b7a139202e0392f1ccfa170be620dd35e/merged 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:05:46.329 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:05:46.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:46.330 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-06-03T01:05:46.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:2024-06-03T01:05:46.442+0000 7f8c21e0b480 -1 init, newargv = 0x56435ef847c0 newargc=15 2024-06-03T01:05:46.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[78407]: starting ceph client 2024-06-03T01:05:46.452 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[78407]: starting fuse 2024-06-03T01:05:47.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:47.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:46 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:47.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:46 smithi107 ceph-mon[36622]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:47.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:47.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:46 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:47.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:46 smithi094 ceph-mon[32119]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:47.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:47.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:46 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:47.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:46 smithi179 ceph-mon[37041]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:05:47.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo modprobe fuse 2024-06-03T01:05:47.612 DEBUG:teuthology.orchestra.run.smithi094:> cat /proc/self/mounts | awk '{print $2}' 2024-06-03T01:05:47.669 INFO:teuthology.orchestra.run.smithi094.stdout:/proc 2024-06-03T01:05:47.669 INFO:teuthology.orchestra.run.smithi094.stdout:/sys 2024-06-03T01:05:47.669 INFO:teuthology.orchestra.run.smithi094.stdout:/dev 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/security 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/shm 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/pts 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/run 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/cgroup 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/pstore 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/bpf 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/ 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/selinux 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T01:05:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/mqueue 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/dev/hugepages 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/debug 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/tracing 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/fs/fuse/connections 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/sys/kernel/config 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-sysctl.service 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/nfs/rpc_pipefs 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/1000 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/ceph 2024-06-03T01:05:47.671 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/a0ab7bf4d975568007299b5077713b847111ebac4c5cb7a2fc755355d7f013f1/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/90d614b5bb1d918ec32f7a1f87529411cefb058a431b01759101d8a3525812a1/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/run/user/0 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/proc/sys/fs/binfmt_misc 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6f0ccb81e202b4cd654a1dc4b16d0cec710790e01594ef3ee41f92c1266e850b/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/8a52d6ca46f848655447f8a4616102989fe09d873b53c9fb3662ed3a82859a15/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/dae867281ca3ecfa954674c2c6daf8fe594b92c75e8540d97eeead8063c3bd6c/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/6a8870a92b7ee7aabf7ad7f5f348508704c6aa4c6be28c278bc34ccacec87a13/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/2590eafe96576f9411553d7d7118b8529eb4290469d78f4417ed7b66283f2435/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/01d99acd279c0cc27a5dd3e648b5e8245db954ed3e78af855619df7bf09183a7/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/cd3d0366dc520e3c5eaf673db688e1891d7cb4639a7cbd0b320e1644c9d92b50/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/var/lib/containers/storage/overlay/4d84f59a7a275a54d905f0784d44984b7a139202e0392f1ccfa170be620dd35e/merged 2024-06-03T01:05:47.672 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns 2024-06-03T01:05:47.673 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:05:47.673 INFO:teuthology.orchestra.run.smithi094.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:05:47.673 INFO:teuthology.orchestra.run.smithi094.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:47.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:47.674 DEBUG:teuthology.orchestra.run.smithi094:> ls /sys/fs/fuse/connections 2024-06-03T01:05:47.724 INFO:teuthology.orchestra.run.smithi094.stdout:119 2024-06-03T01:05:47.725 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-03T01:05:47.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo stdin-killer -- python3 -c ' 2024-06-03T01:05:47.725 DEBUG:teuthology.orchestra.run.smithi094:> import glob 2024-06-03T01:05:47.725 DEBUG:teuthology.orchestra.run.smithi094:> import re 2024-06-03T01:05:47.725 DEBUG:teuthology.orchestra.run.smithi094:> import os 2024-06-03T01:05:47.725 DEBUG:teuthology.orchestra.run.smithi094:> import subprocess 2024-06-03T01:05:47.725 DEBUG:teuthology.orchestra.run.smithi094:> 2024-06-03T01:05:47.725 DEBUG:teuthology.orchestra.run.smithi094:> def _find_admin_socket(client_name): 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> files = glob.glob(asok_path) 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> # Given a non-glob path, it better be there 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> if "*" not in asok_path: 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> assert(len(files) == 1) 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> return files[0] 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> for f in files: 2024-06-03T01:05:47.726 DEBUG:teuthology.orchestra.run.smithi094:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> contents = proc_f.read() 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> if mountpoint in contents: 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> return f 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> print(_find_admin_socket("client.admin")) 2024-06-03T01:05:47.727 DEBUG:teuthology.orchestra.run.smithi094:> ' 2024-06-03T01:05:47.890 INFO:teuthology.orchestra.run.smithi094.stdout:/var/run/ceph/ceph-client.admin.78407.asok 2024-06-03T01:05:47.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:05:47.908 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78407.asok 2024-06-03T01:05:47.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:47.909 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78407.asok status 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout: "metadata": { 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout: "entity_id": "admin", 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": "smithi094", 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-03T01:05:48.140 INFO:teuthology.orchestra.run.smithi094.stdout: "pid": "78407", 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "root": "/" 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_count": 0, 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "dentry_pinned_count": 0, 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24643, 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "inst": { 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": { 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "client", 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "num": 24643 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3468000858 2024-06-03T01:05:48.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": { 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3468000858 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "inst_str": "client.24643 172.21.15.94:0/3468000858", 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "addr_str": "172.21.15.94:0/3468000858", 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "inode_count": 1, 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_epoch": 16, 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch": 98, 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_epoch_barrier": 0, 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklisted": false, 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs" 2024-06-03T01:05:48.142 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:05:48.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:48.155 DEBUG:teuthology.orchestra.run.smithi094:> stat --file-system '--printf=%T 2024-06-03T01:05:48.156 DEBUG:teuthology.orchestra.run.smithi094:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:48.209 INFO:teuthology.orchestra.run.smithi094.stdout:fuseblk 2024-06-03T01:05:48.209 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:48.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:48.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:48.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:05:48.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:05:48.279 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:47 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:48.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:05:48.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:05:48.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:47 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:48.294 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-03T01:05:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:05:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:05:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:47 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:05:48.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:05:48.439 INFO:teuthology.orchestra.run.smithi094.stdout:# file: . 2024-06-03T01:05:48.439 INFO:teuthology.orchestra.run.smithi094.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-03T01:05:48.440 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:05:48.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:05:48.451 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T01:05:48.451 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:05:48.501 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T01:05:48.982 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:49.256 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:48 smithi179 ceph-mon[37041]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:05:49.271 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:48 smithi094 ceph-mon[32119]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:05:49.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:48 smithi107 ceph-mon[36622]: pgmap v365: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:05:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 -- 172.21.15.94:0/378280536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c1076f0 msgr2=0x7fd64c10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 --2- 172.21.15.94:0/378280536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c1076f0 0x7fd64c10f2c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd64400b0d0 tx=0x7fd64400b3e0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 -- 172.21.15.94:0/378280536 shutdown_connections 2024-06-03T01:05:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 --2- 172.21.15.94:0/378280536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c1076f0 0x7fd64c10f2c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 --2- 172.21.15.94:0/378280536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c102610 0x7fd64c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 --2- 172.21.15.94:0/378280536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd64c102fc0 0x7fd64c107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 -- 172.21.15.94:0/378280536 >> 172.21.15.94:0/378280536 conn(0x7fd64c0fdea0 msgr2=0x7fd64c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 -- 172.21.15.94:0/378280536 shutdown_connections 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.315+0000 7fd64b200700 1 -- 172.21.15.94:0/378280536 wait complete. 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.316+0000 7fd64b200700 1 Processor -- start 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.316+0000 7fd64b200700 1 -- start start 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd64b200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd64c102610 0x7fd64c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd64b200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c102fc0 0x7fd64c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd64b200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c1076f0 0x7fd64c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd64b200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c1126e0 con 0x7fd64c102fc0 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd64b200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c112560 con 0x7fd64c102610 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd64b200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c112860 con 0x7fd64c1076f0 2024-06-03T01:05:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c102fc0 0x7fd64c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd64c102610 0x7fd64c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c102fc0 0x7fd64c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35162/0 (socket says 172.21.15.94:35162) 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649400700 1 -- 172.21.15.94:0/3186741112 learned_addr learned my addr 172.21.15.94:0/3186741112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd64a800700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c1076f0 0x7fd64c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649e00700 1 -- 172.21.15.94:0/3186741112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c1076f0 msgr2=0x7fd64c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649e00700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c1076f0 0x7fd64c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649e00700 1 -- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c102fc0 msgr2=0x7fd64c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649e00700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c102fc0 0x7fd64c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649e00700 1 -- 172.21.15.94:0/3186741112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64c1a7030 con 0x7fd64c102610 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.317+0000 7fd649400700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c102fc0 0x7fd64c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.318+0000 7fd649e00700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd64c102610 0x7fd64c074790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00d750 tx=0x7fd63c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.318+0000 7fd642a00700 1 -- 172.21.15.94:0/3186741112 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c014030 con 0x7fd64c102610 2024-06-03T01:05:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.318+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64c1a7320 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.318+0000 7fd642a00700 1 -- 172.21.15.94:0/3186741112 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd63c004500 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.318+0000 7fd642a00700 1 -- 172.21.15.94:0/3186741112 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c01d060 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.318+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64c1a78b0 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.319+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64c069180 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.321+0000 7fd642a00700 1 -- 172.21.15.94:0/3186741112 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd63c004a30 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.323+0000 7fd642a00700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd6300a2c20 0x7fd6300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.323+0000 7fd642a00700 1 -- 172.21.15.94:0/3186741112 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fd63c0cecc0 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.323+0000 7fd649400700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd6300a2c20 0x7fd6300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.324+0000 7fd649400700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd6300a2c20 0x7fd6300a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd64c108ba0 tx=0x7fd638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:51.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.327+0000 7fd642a00700 1 -- 172.21.15.94:0/3186741112 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd63c0d5080 con 0x7fd64c102610 2024-06-03T01:05:51.407 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:51 smithi094 ceph-mon[32119]: pgmap v366: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:05:51.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.568+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd64c02dca0 con 0x7fd64c102610 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.569+0000 7fd642a00700 1 -- 172.21.15.94:0/3186741112 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+589 (secure 0 0 0) 0x7fd63c09b860 con 0x7fd64c102610 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/127652834 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/127652834 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2902022794 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3243526621 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2467202876 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2467202876 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3121579087 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:51.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1105337314 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:51.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3874005975 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:51.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/516090952 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd6300a2c20 msgr2=0x7fd6300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd6300a2c20 0x7fd6300a50e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd64c108ba0 tx=0x7fd638009260 comp rx=0 tx=0).stop 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd64c102610 msgr2=0x7fd64c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd64c102610 0x7fd64c074790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00d750 tx=0x7fd63c00da60 comp rx=0 tx=0).stop 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 shutdown_connections 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd64c102610 0x7fd64c074790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fd6300a2c20 0x7fd6300a50e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd64c1076f0 0x7fd64c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 --2- 172.21.15.94:0/3186741112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd64c102fc0 0x7fd64c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:51.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.572+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 >> 172.21.15.94:0/3186741112 conn(0x7fd64c0fdea0 msgr2=0x7fd64c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:51.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.573+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 shutdown_connections 2024-06-03T01:05:51.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:51.573+0000 7fd64b200700 1 -- 172.21.15.94:0/3186741112 wait complete. 2024-06-03T01:05:51.574 INFO:teuthology.orchestra.run.smithi094.stderr:listed 10 entries 2024-06-03T01:05:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:51 smithi179 ceph-mon[37041]: pgmap v366: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:05:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:51 smithi107 ceph-mon[36622]: pgmap v366: 129 pgs: 129 active+clean; 579 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-03T01:05:52.474 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T01:05:52.474 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:05:52.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T01:05:52.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:52 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3186741112' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:05:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:52 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3186741112' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:05:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:52 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3186741112' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:05:52.962 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:53.372 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:53 smithi094 ceph-mon[32119]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:53.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:53 smithi179 ceph-mon[37041]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:53.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:53 smithi107 ceph-mon[36622]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:54.740 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:54 smithi094 ceph-mon[32119]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:54 smithi107 ceph-mon[36622]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:54 smithi179 ceph-mon[37041]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3441092954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 msgr2=0x7f5bbc10eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3441092954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 0x7f5bbc10eea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0098f0 tx=0x7f5bac009c00 comp rx=0 tx=0).stop 2024-06-03T01:05:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3441092954 shutdown_connections 2024-06-03T01:05:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3441092954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 0x7f5bbc10eea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3441092954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc109480 0x7f5bbc115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3441092954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc108ac0 0x7f5bbc108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3441092954 >> 172.21.15.94:0/3441092954 conn(0x7f5bbc075c60 msgr2=0x7f5bbc1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.288+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3441092954 shutdown_connections 2024-06-03T01:05:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.289+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3441092954 wait complete. 2024-06-03T01:05:55.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.290+0000 7f5bc2c00700 1 Processor -- start 2024-06-03T01:05:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.290+0000 7f5bc2c00700 1 -- start start 2024-06-03T01:05:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.290+0000 7f5bc2c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc108ac0 0x7f5bbc19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.290+0000 7f5bc2c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc109480 0x7f5bbc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.290+0000 7f5bc2c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 0x7f5bbc1a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bc2c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc118e10 con 0x7f5bbc108ac0 2024-06-03T01:05:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bc2c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc118c90 con 0x7f5bbc10eac0 2024-06-03T01:05:55.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bc2c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc118f90 con 0x7f5bbc109480 2024-06-03T01:05:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc108ac0 0x7f5bbc19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc108ac0 0x7f5bbc19f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57940/0 (socket says 172.21.15.94:57940) 2024-06-03T01:05:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 -- 172.21.15.94:0/3552314689 learned_addr learned my addr 172.21.15.94:0/3552314689 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbb400700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc109480 0x7f5bbc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:55.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bc0c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 0x7f5bbc1a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 -- 172.21.15.94:0/3552314689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc109480 msgr2=0x7f5bbc19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc109480 0x7f5bbc19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 -- 172.21.15.94:0/3552314689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 msgr2=0x7f5bbc1a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 0x7f5bbc1a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbbe00700 1 -- 172.21.15.94:0/3552314689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc1a4290 con 0x7f5bbc108ac0 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bbb400700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc109480 0x7f5bbc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.291+0000 7f5bc0c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 0x7f5bbc1a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.292+0000 7f5bbbe00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc108ac0 0x7f5bbc19f280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009230 tx=0x7f5bac00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.292+0000 7f5bb8c00700 1 -- 172.21.15.94:0/3552314689 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac005180 con 0x7f5bbc108ac0 2024-06-03T01:05:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.292+0000 7f5bb8c00700 1 -- 172.21.15.94:0/3552314689 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bac00b560 con 0x7f5bbc108ac0 2024-06-03T01:05:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.292+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc1a4520 con 0x7f5bbc108ac0 2024-06-03T01:05:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.292+0000 7f5bb8c00700 1 -- 172.21.15.94:0/3552314689 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac003a70 con 0x7f5bbc108ac0 2024-06-03T01:05:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.292+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1abe00 con 0x7f5bbc108ac0 2024-06-03T01:05:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.293+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc10a200 con 0x7f5bbc108ac0 2024-06-03T01:05:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.295+0000 7f5bb8c00700 1 -- 172.21.15.94:0/3552314689 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5bac029060 con 0x7f5bbc108ac0 2024-06-03T01:05:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.297+0000 7f5bb8c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5ba00a2c20 0x7f5ba00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.297+0000 7f5bb8c00700 1 -- 172.21.15.94:0/3552314689 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f5bac0cfcc0 con 0x7f5bbc108ac0 2024-06-03T01:05:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.297+0000 7f5bbb400700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5ba00a2c20 0x7f5ba00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:55.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.298+0000 7f5bbb400700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5ba00a2c20 0x7f5ba00a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc077bb0 tx=0x7f5ba8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:55.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.302+0000 7f5bb8c00700 1 -- 172.21.15.94:0/3552314689 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5bac0d6080 con 0x7f5bbc108ac0 2024-06-03T01:05:55.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.543+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5bbc02dca0 con 0x7f5bbc108ac0 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.544+0000 7f5bb8c00700 1 -- 172.21.15.94:0/3552314689 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+589 (secure 0 0 0) 0x7f5bac013030 con 0x7f5bbc108ac0 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/127652834 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/127652834 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2902022794 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3243526621 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2467202876 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2467202876 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:55.544 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3121579087 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:55.545 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1105337314 2024-06-04T00:55:59.358088+0000 2024-06-03T01:05:55.545 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3874005975 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:55.545 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/516090952 2024-06-04T00:56:34.111978+0000 2024-06-03T01:05:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5ba00a2c20 msgr2=0x7f5ba00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5ba00a2c20 0x7f5ba00a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc077bb0 tx=0x7f5ba8009260 comp rx=0 tx=0).stop 2024-06-03T01:05:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc108ac0 msgr2=0x7f5bbc19f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc108ac0 0x7f5bbc19f280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009230 tx=0x7f5bac00fa70 comp rx=0 tx=0).stop 2024-06-03T01:05:55.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 shutdown_connections 2024-06-03T01:05:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bbc10eac0 0x7f5bbc1a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5ba00a2c20 0x7f5ba00a50e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bbc108ac0 0x7f5bbc19f280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 --2- 172.21.15.94:0/3552314689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bbc109480 0x7f5bbc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 >> 172.21.15.94:0/3552314689 conn(0x7f5bbc075c60 msgr2=0x7f5bbc076240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 shutdown_connections 2024-06-03T01:05:55.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:55.547+0000 7f5bc2c00700 1 -- 172.21.15.94:0/3552314689 wait complete. 2024-06-03T01:05:55.549 INFO:teuthology.orchestra.run.smithi094.stderr:listed 10 entries 2024-06-03T01:05:55.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:55 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3552314689' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:05:55.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:55 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3552314689' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:05:56.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:55 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3552314689' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:05:56.498 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi094.front.sepia.ceph.com... 2024-06-03T01:05:56.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:56.499 DEBUG:teuthology.orchestra.run.smithi094:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-03T01:05:56.582 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi094.stderr:ceph-fuse[78407]: fuse finished with error 0 and tester_r 0 2024-06-03T01:05:56.587 INFO:teuthology.orchestra.run:waiting for 300 2024-06-03T01:05:56.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:05:56.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:05:56.646 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-03T01:05:56.712 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:05:56.784 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:56 smithi094 ceph-mon[32119]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:56.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:56 smithi179 ceph-mon[37041]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:57.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:56 smithi107 ceph-mon[36622]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-03T01:05:57.151 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:05:59.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.578+0000 7f4843c00700 1 -- 172.21.15.94:0/880976538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c105070 msgr2=0x7f483c111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:59.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.578+0000 7f4843c00700 1 --2- 172.21.15.94:0/880976538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c105070 0x7f483c111580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f483400b0a0 tx=0x7f483400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.578+0000 7f4843c00700 1 -- 172.21.15.94:0/880976538 shutdown_connections 2024-06-03T01:05:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.578+0000 7f4843c00700 1 --2- 172.21.15.94:0/880976538 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c105070 0x7f483c111580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.578+0000 7f4843c00700 1 --2- 172.21.15.94:0/880976538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c10a6b0 0x7f483c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.578+0000 7f4843c00700 1 --2- 172.21.15.94:0/880976538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1046b0 0x7f483c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.578+0000 7f4843c00700 1 -- 172.21.15.94:0/880976538 >> 172.21.15.94:0/880976538 conn(0x7f483c1001a0 msgr2=0x7f483c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.579+0000 7f4843c00700 1 -- 172.21.15.94:0/880976538 shutdown_connections 2024-06-03T01:05:59.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.579+0000 7f4843c00700 1 -- 172.21.15.94:0/880976538 wait complete. 2024-06-03T01:05:59.580 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:05:59 smithi094 ceph-mon[32119]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 Processor -- start 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 -- start start 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1046b0 0x7f483c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c105070 0x7f483c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c10a6b0 0x7f483c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f483c114a90 con 0x7f483c10a6b0 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f483c114910 con 0x7f483c1046b0 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.580+0000 7f4843c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f483c114c10 con 0x7f483c105070 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c105070 0x7f483c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:59.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f4841c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c10a6b0 0x7f483c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c105070 0x7f483c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55022/0 (socket says 172.21.15.94:55022) 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 -- 172.21.15.94:0/1809103979 learned_addr learned my addr 172.21.15.94:0/1809103979 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f4841200700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1046b0 0x7f483c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 -- 172.21.15.94:0/1809103979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1046b0 msgr2=0x7f483c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1046b0 0x7f483c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 -- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c10a6b0 msgr2=0x7f483c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c10a6b0 0x7f483c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.581+0000 7f483be00700 1 -- 172.21.15.94:0/1809103979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c19fe10 con 0x7f483c105070 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.582+0000 7f483be00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c105070 0x7f483c19b340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f482800eb30 tx=0x7f482800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.582+0000 7f4839600700 1 -- 172.21.15.94:0/1809103979 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482800cbd0 con 0x7f483c105070 2024-06-03T01:05:59.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.582+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c1a0100 con 0x7f483c105070 2024-06-03T01:05:59.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.582+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1a7980 con 0x7f483c105070 2024-06-03T01:05:59.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.582+0000 7f4839600700 1 -- 172.21.15.94:0/1809103979 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4828004500 con 0x7f483c105070 2024-06-03T01:05:59.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.582+0000 7f4839600700 1 -- 172.21.15.94:0/1809103979 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828018430 con 0x7f483c105070 2024-06-03T01:05:59.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.583+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c005330 con 0x7f483c105070 2024-06-03T01:05:59.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.585+0000 7f4839600700 1 -- 172.21.15.94:0/1809103979 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4828003680 con 0x7f483c105070 2024-06-03T01:05:59.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.586+0000 7f4839600700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f48240a2ca0 0x7f48240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:05:59.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.586+0000 7f4839600700 1 -- 172.21.15.94:0/1809103979 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f48280cf5a0 con 0x7f483c105070 2024-06-03T01:05:59.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.587+0000 7f4841200700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f48240a2ca0 0x7f48240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:05:59.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.587+0000 7f4841200700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f48240a2ca0 0x7f48240a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4830009be0 tx=0x7f4830009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:05:59.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.591+0000 7f4839600700 1 -- 172.21.15.94:0/1809103979 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f482809c140 con 0x7f483c105070 2024-06-03T01:05:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:05:59 smithi179 ceph-mon[37041]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:05:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:05:59 smithi107 ceph-mon[36622]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-03T01:05:59.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.816+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f481c005620 con 0x7f483c105070 2024-06-03T01:05:59.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.818+0000 7f4839600700 1 -- 172.21.15.94:0/1809103979 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24655 (secure 0 0 0) 0x7f482809ba40 con 0x7f483c105070 2024-06-03T01:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:05:59.819 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:05:35.239124+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:05:59.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f48240a2ca0 msgr2=0x7f48240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f48240a2ca0 0x7f48240a5160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4830009be0 tx=0x7f4830009260 comp rx=0 tx=0).stop 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c105070 msgr2=0x7f483c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c105070 0x7f483c19b340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f482800eb30 tx=0x7f482800ee40 comp rx=0 tx=0).stop 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 shutdown_connections 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1046b0 0x7f483c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f48240a2ca0 0x7f48240a5160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f483c10a6b0 0x7f483c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 --2- 172.21.15.94:0/1809103979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f483c105070 0x7f483c19b340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:05:59.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.821+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 >> 172.21.15.94:0/1809103979 conn(0x7f483c1001a0 msgr2=0x7f483c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:05:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.822+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 shutdown_connections 2024-06-03T01:05:59.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:05:59.822+0000 7f4843c00700 1 -- 172.21.15.94:0/1809103979 wait complete. 2024-06-03T01:06:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:00 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1809103979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:00 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1809103979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:00 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1809103979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:00.790 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:06:01.226 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:01 smithi179 ceph-mon[37041]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-03T01:06:01.613 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:01 smithi094 ceph-mon[32119]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-03T01:06:01.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:01 smithi107 ceph-mon[36622]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-03T01:06:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:02 smithi094 ceph-mon[32119]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-03T01:06:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:02 smithi107 ceph-mon[36622]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-03T01:06:02.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:02 smithi179 ceph-mon[37041]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 -- 172.21.15.94:0/3901792774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774109460 msgr2=0x7ff774115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 --2- 172.21.15.94:0/3901792774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774109460 0x7ff774115970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b0a0 tx=0x7ff76c00b3b0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 -- 172.21.15.94:0/3901792774 shutdown_connections 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 --2- 172.21.15.94:0/3901792774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77410eaa0 0x7ff77410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 --2- 172.21.15.94:0/3901792774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774109460 0x7ff774115970 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 --2- 172.21.15.94:0/3901792774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff774108aa0 0x7ff774108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 -- 172.21.15.94:0/3901792774 >> 172.21.15.94:0/3901792774 conn(0x7ff774075c60 msgr2=0x7ff7741072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:03.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 -- 172.21.15.94:0/3901792774 shutdown_connections 2024-06-03T01:06:03.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.646+0000 7ff77a600700 1 -- 172.21.15.94:0/3901792774 wait complete. 2024-06-03T01:06:03.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 Processor -- start 2024-06-03T01:06:03.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 -- start start 2024-06-03T01:06:03.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774108aa0 0x7ff77419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:03.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff774109460 0x7ff77419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:03.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77410eaa0 0x7ff7741a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:03.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff774118e40 con 0x7ff774108aa0 2024-06-03T01:06:03.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff774118cc0 con 0x7ff77410eaa0 2024-06-03T01:06:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.648+0000 7ff77a600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff774118fc0 con 0x7ff774109460 2024-06-03T01:06:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774108aa0 0x7ff77419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774108aa0 0x7ff77419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57976/0 (socket says 172.21.15.94:57976) 2024-06-03T01:06:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 -- 172.21.15.94:0/2392514070 learned_addr learned my addr 172.21.15.94:0/2392514070 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773e00700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77410eaa0 0x7ff7741a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:03.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff772a00700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff774109460 0x7ff77419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:03.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 -- 172.21.15.94:0/2392514070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff774109460 msgr2=0x7ff77419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:03.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff774109460 0x7ff77419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 -- 172.21.15.94:0/2392514070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77410eaa0 msgr2=0x7ff7741a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:03.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77410eaa0 0x7ff7741a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773400700 1 -- 172.21.15.94:0/2392514070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7741a41e0 con 0x7ff774108aa0 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.649+0000 7ff773e00700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77410eaa0 0x7ff7741a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.650+0000 7ff773400700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774108aa0 0x7ff77419f1d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d950 tx=0x7ff76400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.650+0000 7ff76be00700 1 -- 172.21.15.94:0/2392514070 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764016070 con 0x7ff774108aa0 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.650+0000 7ff76be00700 1 -- 172.21.15.94:0/2392514070 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff76400fd70 con 0x7ff774108aa0 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.650+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7741a44d0 con 0x7ff774108aa0 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.650+0000 7ff76be00700 1 -- 172.21.15.94:0/2392514070 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76400e610 con 0x7ff774108aa0 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.650+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7741abd50 con 0x7ff774108aa0 2024-06-03T01:06:03.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.651+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff774069180 con 0x7ff774108aa0 2024-06-03T01:06:03.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.653+0000 7ff76be00700 1 -- 172.21.15.94:0/2392514070 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff764019070 con 0x7ff774108aa0 2024-06-03T01:06:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.654+0000 7ff76be00700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff7580a2c20 0x7ff7580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.655+0000 7ff76be00700 1 -- 172.21.15.94:0/2392514070 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff7640cee00 con 0x7ff774108aa0 2024-06-03T01:06:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.655+0000 7ff772a00700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff7580a2c20 0x7ff7580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.656+0000 7ff772a00700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff7580a2c20 0x7ff7580a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff774077a50 tx=0x7ff760009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:03.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.658+0000 7ff76be00700 1 -- 172.21.15.94:0/2392514070 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff76409b9a0 con 0x7ff774108aa0 2024-06-03T01:06:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.882+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7741abfe0 con 0x7ff774108aa0 2024-06-03T01:06:03.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.884+0000 7ff76be00700 1 -- 172.21.15.94:0/2392514070 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24655 (secure 0 0 0) 0x7ff76409b2a0 con 0x7ff774108aa0 2024-06-03T01:06:03.885 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:06:03.885 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":98,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:05:35.239124+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:06:03.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.887+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff7580a2c20 msgr2=0x7ff7580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:03.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.887+0000 7ff77a600700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff7580a2c20 0x7ff7580a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff774077a50 tx=0x7ff760009260 comp rx=0 tx=0).stop 2024-06-03T01:06:03.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.887+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774108aa0 msgr2=0x7ff77419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:03.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.887+0000 7ff77a600700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774108aa0 0x7ff77419f1d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d950 tx=0x7ff76400dc60 comp rx=0 tx=0).stop 2024-06-03T01:06:03.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 shutdown_connections 2024-06-03T01:06:03.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77410eaa0 0x7ff7741a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff7580a2c20 0x7ff7580a50e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff774108aa0 0x7ff77419f1d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 --2- 172.21.15.94:0/2392514070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff774109460 0x7ff77419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:03.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 >> 172.21.15.94:0/2392514070 conn(0x7ff774075c60 msgr2=0x7ff7740760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:03.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 shutdown_connections 2024-06-03T01:06:03.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:03.888+0000 7ff77a600700 1 -- 172.21.15.94:0/2392514070 wait complete. 2024-06-03T01:06:04.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:03 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2392514070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:04.315 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:03 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2392514070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:04.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:03 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2392514070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:04.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-03T01:06:05.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:04 smithi094 ceph-mon[32119]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:05.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:04 smithi107 ceph-mon[36622]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:05.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:04 smithi179 ceph-mon[37041]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:07.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:07 smithi094 ceph-mon[32119]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:07 smithi179 ceph-mon[37041]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:07.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 -- 172.21.15.94:0/3769542274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c102ea0 msgr2=0x7f8f6c10f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:07.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 --2- 172.21.15.94:0/3769542274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c102ea0 0x7f8f6c10f380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400b0a0 tx=0x7f8f6400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:06:07.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 -- 172.21.15.94:0/3769542274 shutdown_connections 2024-06-03T01:06:07.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 --2- 172.21.15.94:0/3769542274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f6c1085f0 0x7f8f6c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 --2- 172.21.15.94:0/3769542274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c102ea0 0x7f8f6c10f380 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 --2- 172.21.15.94:0/3769542274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c1024e0 0x7f8f6c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 -- 172.21.15.94:0/3769542274 >> 172.21.15.94:0/3769542274 conn(0x7f8f6c079640 msgr2=0x7f8f6c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 -- 172.21.15.94:0/3769542274 shutdown_connections 2024-06-03T01:06:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.646+0000 7f8f72400700 1 -- 172.21.15.94:0/3769542274 wait complete. 2024-06-03T01:06:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.647+0000 7f8f72400700 1 Processor -- start 2024-06-03T01:06:07.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.647+0000 7f8f72400700 1 -- start start 2024-06-03T01:06:07.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f72400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c1024e0 0x7f8f6c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:07.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f72400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f6c102ea0 0x7f8f6c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f72400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c1085f0 0x7f8f6c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f72400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c112800 con 0x7f8f6c1024e0 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f72400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c112680 con 0x7f8f6c102ea0 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f72400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c112980 con 0x7f8f6c1085f0 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f6b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c1024e0 0x7f8f6c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f63e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f6c102ea0 0x7f8f6c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f6b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c1024e0 0x7f8f6c19ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36244/0 (socket says 172.21.15.94:36244) 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f6b400700 1 -- 172.21.15.94:0/2971866264 learned_addr learned my addr 172.21.15.94:0/2971866264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:07.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.648+0000 7f8f6be00700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c1085f0 0x7f8f6c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.649+0000 7f8f63e00700 1 -- 172.21.15.94:0/2971866264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c1085f0 msgr2=0x7f8f6c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.649+0000 7f8f63e00700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c1085f0 0x7f8f6c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.649+0000 7f8f63e00700 1 -- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c1024e0 msgr2=0x7f8f6c19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.649+0000 7f8f63e00700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c1024e0 0x7f8f6c19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.649+0000 7f8f63e00700 1 -- 172.21.15.94:0/2971866264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c19fd80 con 0x7f8f6c102ea0 2024-06-03T01:06:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.649+0000 7f8f63e00700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f6c102ea0 0x7f8f6c19b2b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e9f0 tx=0x7f8f5400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:07.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.649+0000 7f8f68c00700 1 -- 172.21.15.94:0/2971866264 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400c9c0 con 0x7f8f6c102ea0 2024-06-03T01:06:07.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.650+0000 7f8f68c00700 1 -- 172.21.15.94:0/2971866264 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f5400cb60 con 0x7f8f6c102ea0 2024-06-03T01:06:07.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.650+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c1a0070 con 0x7f8f6c102ea0 2024-06-03T01:06:07.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.650+0000 7f8f68c00700 1 -- 172.21.15.94:0/2971866264 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54018680 con 0x7f8f6c102ea0 2024-06-03T01:06:07.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.650+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1a78f0 con 0x7f8f6c102ea0 2024-06-03T01:06:07.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.651+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c069180 con 0x7f8f6c102ea0 2024-06-03T01:06:07.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.653+0000 7f8f68c00700 1 -- 172.21.15.94:0/2971866264 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f54010430 con 0x7f8f6c102ea0 2024-06-03T01:06:07.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.654+0000 7f8f68c00700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8f500a2c20 0x7f8f500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:07.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.654+0000 7f8f68c00700 1 -- 172.21.15.94:0/2971866264 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f8f540cf510 con 0x7f8f6c102ea0 2024-06-03T01:06:07.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.654+0000 7f8f6b400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8f500a2c20 0x7f8f500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:07.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.655+0000 7f8f6b400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8f500a2c20 0x7f8f500a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009d60 tx=0x7f8f5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:07.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.658+0000 7f8f68c00700 1 -- 172.21.15.94:0/2971866264 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f5409c0b0 con 0x7f8f6c102ea0 2024-06-03T01:06:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:07 smithi107 ceph-mon[36622]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:07.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:07.934+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8f6c1a7af0 con 0x7f8f6c102ea0 2024-06-03T01:06:08.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.362+0000 7f8f68c00700 1 -- 172.21.15.94:0/2971866264 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f8f5409b9b0 con 0x7f8f6c102ea0 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8f500a2c20 msgr2=0x7f8f500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8f500a2c20 0x7f8f500a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009d60 tx=0x7f8f5c009260 comp rx=0 tx=0).stop 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f6c102ea0 msgr2=0x7f8f6c19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f6c102ea0 0x7f8f6c19b2b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400e9f0 tx=0x7f8f5400ed00 comp rx=0 tx=0).stop 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 shutdown_connections 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f8f500a2c20 0x7f8f500a50e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f6c1085f0 0x7f8f6c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f6c1024e0 0x7f8f6c19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:08.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 --2- 172.21.15.94:0/2971866264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f6c102ea0 0x7f8f6c19b2b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.365+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 >> 172.21.15.94:0/2971866264 conn(0x7f8f6c079640 msgr2=0x7f8f6c0746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.366+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 shutdown_connections 2024-06-03T01:06:08.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:08.366+0000 7f8f72400700 1 -- 172.21.15.94:0/2971866264 wait complete. 2024-06-03T01:06:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:08 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2971866264' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T01:06:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:08 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T01:06:08.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:08 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2971866264' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T01:06:08.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:08 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T01:06:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:08 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2971866264' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T01:06:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:08 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-03T01:06:09.479 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-03T01:06:09.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-03T01:06:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:09 smithi179 ceph-mon[37041]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:09 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T01:06:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:09 smithi179 ceph-mon[37041]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:06:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:09 smithi107 ceph-mon[36622]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:09 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T01:06:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:09 smithi107 ceph-mon[36622]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:06:09.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:09 smithi094 ceph-mon[32119]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:09.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:09 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-03T01:06:09.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:09 smithi094 ceph-mon[32119]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-03T01:06:09.908 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:10.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:10 smithi094 ceph-mon[32119]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:06:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:10 smithi107 ceph-mon[36622]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:06:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:10 smithi179 ceph-mon[37041]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.204+0000 7f6ba9c00700 1 -- 172.21.15.94:0/1618261306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 msgr2=0x7f6ba4107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.204+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/1618261306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 0x7f6ba4107830 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009990 tx=0x7f6b94009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.204+0000 7f6ba9c00700 1 -- 172.21.15.94:0/1618261306 shutdown_connections 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.204+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/1618261306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 0x7f6ba410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.204+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/1618261306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 0x7f6ba4107830 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.204+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/1618261306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ba4102540 0x7f6ba41029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.204+0000 7f6ba9c00700 1 -- 172.21.15.94:0/1618261306 >> 172.21.15.94:0/1618261306 conn(0x7f6ba40795f0 msgr2=0x7f6ba4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.205+0000 7f6ba9c00700 1 -- 172.21.15.94:0/1618261306 shutdown_connections 2024-06-03T01:06:12.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.205+0000 7f6ba9c00700 1 -- 172.21.15.94:0/1618261306 wait complete. 2024-06-03T01:06:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 Processor -- start 2024-06-03T01:06:12.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 -- start start 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ba4102540 0x7f6ba419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 0x7f6ba419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 0x7f6ba419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4112830 con 0x7f6ba4107450 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41126b0 con 0x7f6ba4102540 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba9c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41129b0 con 0x7f6ba4102ee0 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba2000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 0x7f6ba419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba2000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 0x7f6ba419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49798/0 (socket says 172.21.15.94:49798) 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba2000700 1 -- 172.21.15.94:0/2682355882 learned_addr learned my addr 172.21.15.94:0/2682355882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.206+0000 7f6ba3400700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 0x7f6ba419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:12.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.207+0000 7f6ba2a00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ba4102540 0x7f6ba419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:12.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.207+0000 7f6ba2000700 1 -- 172.21.15.94:0/2682355882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ba4102540 msgr2=0x7f6ba419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:12.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.207+0000 7f6ba2000700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ba4102540 0x7f6ba419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.207+0000 7f6ba2000700 1 -- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 msgr2=0x7f6ba419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.207+0000 7f6ba2000700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 0x7f6ba419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.207+0000 7f6ba2000700 1 -- 172.21.15.94:0/2682355882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba419fed0 con 0x7f6ba4102ee0 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.207+0000 7f6ba2000700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 0x7f6ba419b400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e9a0 tx=0x7f6b9000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.208+0000 7f6b9b400700 1 -- 172.21.15.94:0/2682355882 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000ca10 con 0x7f6ba4102ee0 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.208+0000 7f6b9b400700 1 -- 172.21.15.94:0/2682355882 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9000cbb0 con 0x7f6ba4102ee0 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.208+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba41a01c0 con 0x7f6ba4102ee0 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.208+0000 7f6b9b400700 1 -- 172.21.15.94:0/2682355882 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90018680 con 0x7f6ba4102ee0 2024-06-03T01:06:12.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.208+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41a7a40 con 0x7f6ba4102ee0 2024-06-03T01:06:12.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.209+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba4069180 con 0x7f6ba4102ee0 2024-06-03T01:06:12.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.212+0000 7f6b9b400700 1 -- 172.21.15.94:0/2682355882 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6b90010430 con 0x7f6ba4102ee0 2024-06-03T01:06:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.213+0000 7f6b9b400700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6b880a2ca0 0x7f6b880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.213+0000 7f6b9b400700 1 -- 172.21.15.94:0/2682355882 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f6b900cfa70 con 0x7f6ba4102ee0 2024-06-03T01:06:12.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.213+0000 7f6ba2a00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6b880a2ca0 0x7f6b880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.214+0000 7f6ba2a00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6b880a2ca0 0x7f6b880a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b94006010 tx=0x7f6b9400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:12.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.214+0000 7f6b9b400700 1 -- 172.21.15.94:0/2682355882 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6b9009c610 con 0x7f6ba4102ee0 2024-06-03T01:06:12.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:12.495+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6ba41a7ca0 con 0x7f6ba4102ee0 2024-06-03T01:06:13.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.329+0000 7f6b9b400700 1 -- 172.21.15.94:0/2682355882 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f6b9009bf10 con 0x7f6ba4102ee0 2024-06-03T01:06:13.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.334+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6b880a2ca0 msgr2=0x7f6b880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:13.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.334+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6b880a2ca0 0x7f6b880a5160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b94006010 tx=0x7f6b9400b260 comp rx=0 tx=0).stop 2024-06-03T01:06:13.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.334+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 msgr2=0x7f6ba419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:13.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.334+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 0x7f6ba419b400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e9a0 tx=0x7f6b9000ecb0 comp rx=0 tx=0).stop 2024-06-03T01:06:13.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.335+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 shutdown_connections 2024-06-03T01:06:13.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.335+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ba4102540 0x7f6ba419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:13.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.335+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6b880a2ca0 0x7f6b880a5160 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:13.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.335+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ba4107450 0x7f6ba419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:13.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.335+0000 7f6ba9c00700 1 --2- 172.21.15.94:0/2682355882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ba4102ee0 0x7f6ba419b400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:13.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.335+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 >> 172.21.15.94:0/2682355882 conn(0x7f6ba40795f0 msgr2=0x7f6ba41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:13.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.336+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 shutdown_connections 2024-06-03T01:06:13.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:13.336+0000 7f6ba9c00700 1 -- 172.21.15.94:0/2682355882 wait complete. 2024-06-03T01:06:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:06:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2682355882' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T01:06:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T01:06:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T01:06:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T01:06:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T01:06:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-03T01:06:13.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:13 smithi094 ceph-mon[32119]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2682355882' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-03T01:06:13.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:13 smithi179 ceph-mon[37041]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2682355882' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-03T01:06:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:13 smithi107 ceph-mon[36622]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-03T01:06:14.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-03T01:06:14.760 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:15 smithi094 ceph-mon[32119]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:15 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] up:active 2024-06-03T01:06:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:15 smithi094 ceph-mon[32119]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T01:06:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:15 smithi094 ceph-mon[32119]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-03T01:06:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:15 smithi094 ceph-mon[32119]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-06-03T01:06:15.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:15 smithi094 ceph-mon[32119]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-03T01:06:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:15 smithi179 ceph-mon[37041]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:15 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] up:active 2024-06-03T01:06:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:15 smithi179 ceph-mon[37041]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T01:06:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:15 smithi179 ceph-mon[37041]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-03T01:06:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:15 smithi179 ceph-mon[37041]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-06-03T01:06:15.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:15 smithi179 ceph-mon[37041]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-03T01:06:15.783 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: 2024-06-03T01:06:15.365+0000 7fc5fe11d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-06-03T01:06:15.783 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: 2024-06-03T01:06:15.365+0000 7fc5fe11d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-03T01:06:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-mon[36622]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-03T01:06:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] up:active 2024-06-03T01:06:15.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-mon[36622]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-03T01:06:15.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-mon[36622]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-03T01:06:15.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-mon[36622]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-06-03T01:06:15.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:15 smithi107 ceph-mon[36622]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-06-03T01:06:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:16 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] up:active 2024-06-03T01:06:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:16 smithi094 ceph-mon[32119]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T01:06:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:16 smithi094 ceph-mon[32119]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-03T01:06:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:16 smithi094 ceph-mon[32119]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-06-03T01:06:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:16 smithi094 ceph-mon[32119]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-06-03T01:06:16.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:16 smithi094 ceph-mon[32119]: pgmap v379: 129 pgs: 129 active+clean; 583 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-03T01:06:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:16 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] up:active 2024-06-03T01:06:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:16 smithi107 ceph-mon[36622]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T01:06:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:16 smithi107 ceph-mon[36622]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-03T01:06:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:16 smithi107 ceph-mon[36622]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-06-03T01:06:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:16 smithi107 ceph-mon[36622]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-06-03T01:06:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:16 smithi107 ceph-mon[36622]: pgmap v379: 129 pgs: 129 active+clean; 583 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-03T01:06:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:16 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] up:active 2024-06-03T01:06:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:16 smithi179 ceph-mon[37041]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-03T01:06:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:16 smithi179 ceph-mon[37041]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-06-03T01:06:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:16 smithi179 ceph-mon[37041]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-06-03T01:06:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:16 smithi179 ceph-mon[37041]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-06-03T01:06:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:16 smithi179 ceph-mon[37041]: pgmap v379: 129 pgs: 129 active+clean; 583 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-03T01:06:17.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 -- 172.21.15.94:0/3077197690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a010a6b0 msgr2=0x7f24a010aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:17.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 --2- 172.21.15.94:0/3077197690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a010a6b0 0x7f24a010aa90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2490009990 tx=0x7f2490009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 -- 172.21.15.94:0/3077197690 shutdown_connections 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 --2- 172.21.15.94:0/3077197690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24a0105070 0x7f24a0111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 --2- 172.21.15.94:0/3077197690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a010a6b0 0x7f24a010aa90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 --2- 172.21.15.94:0/3077197690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24a01046b0 0x7f24a0104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 -- 172.21.15.94:0/3077197690 >> 172.21.15.94:0/3077197690 conn(0x7f24a0100160 msgr2=0x7f24a0102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 -- 172.21.15.94:0/3077197690 shutdown_connections 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.118+0000 7f24a6c00700 1 -- 172.21.15.94:0/3077197690 wait complete. 2024-06-03T01:06:17.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.119+0000 7f24a6c00700 1 Processor -- start 2024-06-03T01:06:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f24a6c00700 1 -- start start 2024-06-03T01:06:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f24a6c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24a01046b0 0x7f24a019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f24a6c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a0105070 0x7f24a019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f24a6c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24a010a6b0 0x7f24a019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f24a6c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0114a00 con 0x7f24a0105070 2024-06-03T01:06:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f24a6c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0114880 con 0x7f24a010a6b0 2024-06-03T01:06:17.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f24a6c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0114b80 con 0x7f24a01046b0 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f249f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a0105070 0x7f24a019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f249fe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24a01046b0 0x7f24a019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f249f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a0105070 0x7f24a019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44604/0 (socket says 172.21.15.94:44604) 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.120+0000 7f249f400700 1 -- 172.21.15.94:0/2591742676 learned_addr learned my addr 172.21.15.94:0/2591742676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f24a4c00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24a010a6b0 0x7f24a019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f249fe00700 1 -- 172.21.15.94:0/2591742676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24a010a6b0 msgr2=0x7f24a019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f249fe00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24a010a6b0 0x7f24a019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f249fe00700 1 -- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a0105070 msgr2=0x7f24a019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f249fe00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a0105070 0x7f24a019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f249fe00700 1 -- 172.21.15.94:0/2591742676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a019fe30 con 0x7f24a01046b0 2024-06-03T01:06:17.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f249fe00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24a01046b0 0x7f24a019ae70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2490006010 tx=0x7f249000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.121+0000 7f249cc00700 1 -- 172.21.15.94:0/2591742676 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490027070 con 0x7f24a01046b0 2024-06-03T01:06:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.122+0000 7f249cc00700 1 -- 172.21.15.94:0/2591742676 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24900056d0 con 0x7f24a01046b0 2024-06-03T01:06:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.122+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a01a00c0 con 0x7f24a01046b0 2024-06-03T01:06:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.122+0000 7f249cc00700 1 -- 172.21.15.94:0/2591742676 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490022d90 con 0x7f24a01046b0 2024-06-03T01:06:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.122+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a01a79a0 con 0x7f24a01046b0 2024-06-03T01:06:17.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.123+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a0069180 con 0x7f24a01046b0 2024-06-03T01:06:17.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.125+0000 7f249cc00700 1 -- 172.21.15.94:0/2591742676 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f249002a070 con 0x7f24a01046b0 2024-06-03T01:06:17.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.126+0000 7f249cc00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24840a2c20 0x7f24840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:17.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.126+0000 7f249cc00700 1 -- 172.21.15.94:0/2591742676 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f24900d0b00 con 0x7f24a01046b0 2024-06-03T01:06:17.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.127+0000 7f249f400700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24840a2c20 0x7f24840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:17.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.127+0000 7f249f400700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24840a2c20 0x7f24840a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f248c009680 tx=0x7f248c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:17.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.129+0000 7f249cc00700 1 -- 172.21.15.94:0/2591742676 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2490031020 con 0x7f24a01046b0 2024-06-03T01:06:17.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:17.400+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f24a002dca0 con 0x7f24a01046b0 2024-06-03T01:06:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:17 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] up:active 2024-06-03T01:06:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:17 smithi107 ceph-mon[36622]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T01:06:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:17 smithi107 ceph-mon[36622]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T01:06:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:17 smithi107 ceph-mon[36622]: Cluster is now healthy 2024-06-03T01:06:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:17 smithi107 ceph-mon[36622]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-06-03T01:06:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:17 smithi107 ceph-mon[36622]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T01:06:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:17 smithi107 ceph-mon[36622]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-03T01:06:17.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:17 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] up:active 2024-06-03T01:06:17.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:17 smithi094 ceph-mon[32119]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T01:06:17.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:17 smithi094 ceph-mon[32119]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T01:06:17.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:17 smithi094 ceph-mon[32119]: Cluster is now healthy 2024-06-03T01:06:17.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:17 smithi094 ceph-mon[32119]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-06-03T01:06:17.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:17 smithi094 ceph-mon[32119]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T01:06:17.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:17 smithi094 ceph-mon[32119]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-03T01:06:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:17 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] up:active 2024-06-03T01:06:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:17 smithi179 ceph-mon[37041]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-03T01:06:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:17 smithi179 ceph-mon[37041]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-03T01:06:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:17 smithi179 ceph-mon[37041]: Cluster is now healthy 2024-06-03T01:06:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:17 smithi179 ceph-mon[37041]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-06-03T01:06:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:17 smithi179 ceph-mon[37041]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-03T01:06:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:17 smithi179 ceph-mon[37041]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-06-03T01:06:18.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.331+0000 7f249cc00700 1 -- 172.21.15.94:0/2591742676 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f2490013070 con 0x7f24a01046b0 2024-06-03T01:06:18.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24840a2c20 msgr2=0x7f24840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:18.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24840a2c20 0x7f24840a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f248c009680 tx=0x7f248c009260 comp rx=0 tx=0).stop 2024-06-03T01:06:18.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24a01046b0 msgr2=0x7f24a019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:18.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24a01046b0 0x7f24a019ae70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2490006010 tx=0x7f249000bab0 comp rx=0 tx=0).stop 2024-06-03T01:06:18.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 shutdown_connections 2024-06-03T01:06:18.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24a010a6b0 0x7f24a019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:18.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24840a2c20 0x7f24840a50e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:18.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24a01046b0 0x7f24a019ae70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:18.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 --2- 172.21.15.94:0/2591742676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a0105070 0x7f24a019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:18.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.336+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 >> 172.21.15.94:0/2591742676 conn(0x7f24a0100160 msgr2=0x7f24a0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:18.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.337+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 shutdown_connections 2024-06-03T01:06:18.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:18.337+0000 7f24a6c00700 1 -- 172.21.15.94:0/2591742676 wait complete. 2024-06-03T01:06:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:18 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] up:active 2024-06-03T01:06:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:18 smithi094 ceph-mon[32119]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T01:06:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:18 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2591742676' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T01:06:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:18 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T01:06:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:18 smithi094 ceph-mon[32119]: pgmap v380: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-06-03T01:06:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:18 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T01:06:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:18 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:06:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:18 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] up:active 2024-06-03T01:06:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:18 smithi107 ceph-mon[36622]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T01:06:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:18 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2591742676' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T01:06:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:18 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T01:06:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:18 smithi107 ceph-mon[36622]: pgmap v380: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-06-03T01:06:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:18 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T01:06:18.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:18 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:06:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:18 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] up:active 2024-06-03T01:06:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:18 smithi179 ceph-mon[37041]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-03T01:06:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:18 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2591742676' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T01:06:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:18 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-03T01:06:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:18 smithi179 ceph-mon[37041]: pgmap v380: 129 pgs: 129 active+clean; 584 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 7 op/s 2024-06-03T01:06:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:18 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-03T01:06:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:18 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:06:19.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-03T01:06:19.780 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:21.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:21 smithi094 ceph-mon[32119]: pgmap v381: 129 pgs: 129 active+clean; 586 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-06-03T01:06:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:21 smithi107 ceph-mon[36622]: pgmap v381: 129 pgs: 129 active+clean; 586 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-06-03T01:06:21.834 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:21 smithi179 ceph-mon[37041]: pgmap v381: 129 pgs: 129 active+clean; 586 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-06-03T01:06:22.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 -- 172.21.15.94:0/226257073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 msgr2=0x7fb6d410ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/226257073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 0x7fb6d410ee80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009990 tx=0x7fb6c8009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 -- 172.21.15.94:0/226257073 shutdown_connections 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/226257073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 0x7fb6d410ee80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/226257073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 0x7fb6d4115970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/226257073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4108aa0 0x7fb6d4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 -- 172.21.15.94:0/226257073 >> 172.21.15.94:0/226257073 conn(0x7fb6d4075c60 msgr2=0x7fb6d41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 -- 172.21.15.94:0/226257073 shutdown_connections 2024-06-03T01:06:22.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.285+0000 7fb6dbc00700 1 -- 172.21.15.94:0/226257073 wait complete. 2024-06-03T01:06:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.286+0000 7fb6dbc00700 1 Processor -- start 2024-06-03T01:06:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.286+0000 7fb6dbc00700 1 -- start start 2024-06-03T01:06:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6dbc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4108aa0 0x7fb6d419f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6dbc00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 0x7fb6d419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:22.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6dbc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 0x7fb6d41a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6dbc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4118df0 con 0x7fb6d4108aa0 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6dbc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4118c70 con 0x7fb6d410eaa0 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6dbc00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4118f70 con 0x7fb6d4109460 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d3e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 0x7fb6d419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d9200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4108aa0 0x7fb6d419f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d9c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 0x7fb6d41a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d3e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 0x7fb6d419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43530/0 (socket says 172.21.15.94:43530) 2024-06-03T01:06:22.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d3e00700 1 -- 172.21.15.94:0/3742252804 learned_addr learned my addr 172.21.15.94:0/3742252804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d3e00700 1 -- 172.21.15.94:0/3742252804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 msgr2=0x7fb6d41a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d3e00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 0x7fb6d41a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d3e00700 1 -- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4108aa0 msgr2=0x7fb6d419f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.287+0000 7fb6d3e00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4108aa0 0x7fb6d419f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.288+0000 7fb6d3e00700 1 -- 172.21.15.94:0/3742252804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41a41c0 con 0x7fb6d4109460 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.288+0000 7fb6d3e00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 0x7fb6d419f740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000d950 tx=0x7fb6c000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.288+0000 7fb6d1600700 1 -- 172.21.15.94:0/3742252804 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0016070 con 0x7fb6d4109460 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.288+0000 7fb6d1600700 1 -- 172.21.15.94:0/3742252804 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c000fd70 con 0x7fb6d4109460 2024-06-03T01:06:22.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.288+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41a44b0 con 0x7fb6d4109460 2024-06-03T01:06:22.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.288+0000 7fb6d1600700 1 -- 172.21.15.94:0/3742252804 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000e610 con 0x7fb6d4109460 2024-06-03T01:06:22.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.288+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41abd30 con 0x7fb6d4109460 2024-06-03T01:06:22.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.290+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d4111030 con 0x7fb6d4109460 2024-06-03T01:06:22.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.291+0000 7fb6d1600700 1 -- 172.21.15.94:0/3742252804 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb6c0019070 con 0x7fb6d4109460 2024-06-03T01:06:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.293+0000 7fb6d1600700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb6bc0a2c20 0x7fb6bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.293+0000 7fb6d1600700 1 -- 172.21.15.94:0/3742252804 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fb6c00ceeb0 con 0x7fb6d4109460 2024-06-03T01:06:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.293+0000 7fb6d9200700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb6bc0a2c20 0x7fb6bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:22.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.294+0000 7fb6d9200700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb6bc0a2c20 0x7fb6bc0a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80098e0 tx=0x7fb6c8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:22.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.297+0000 7fb6d1600700 1 -- 172.21.15.94:0/3742252804 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb6c009ba50 con 0x7fb6d4109460 2024-06-03T01:06:22.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:22.584+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb6d41a0510 con 0x7fb6d4109460 2024-06-03T01:06:22.584 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:22 smithi094 ceph-mon[32119]: pgmap v382: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T01:06:22.756 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:22 smithi179 ceph-mon[37041]: pgmap v382: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T01:06:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:22 smithi107 ceph-mon[36622]: pgmap v382: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T01:06:23.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.378+0000 7fb6d1600700 1 -- 172.21.15.94:0/3742252804 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fb6c009b350 con 0x7fb6d4109460 2024-06-03T01:06:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.383+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb6bc0a2c20 msgr2=0x7fb6bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:23.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.383+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb6bc0a2c20 0x7fb6bc0a50e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80098e0 tx=0x7fb6c8005960 comp rx=0 tx=0).stop 2024-06-03T01:06:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.383+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 msgr2=0x7fb6d419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.383+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 0x7fb6d419f740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000d950 tx=0x7fb6c000dc60 comp rx=0 tx=0).stop 2024-06-03T01:06:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.383+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 shutdown_connections 2024-06-03T01:06:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.383+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d410eaa0 0x7fb6d41a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.383+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb6bc0a2c20 0x7fb6bc0a50e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.384+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6d4108aa0 0x7fb6d419f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:23.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.384+0000 7fb6dbc00700 1 --2- 172.21.15.94:0/3742252804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6d4109460 0x7fb6d419f740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.384+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 >> 172.21.15.94:0/3742252804 conn(0x7fb6d4075c60 msgr2=0x7fb6d4076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.384+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 shutdown_connections 2024-06-03T01:06:23.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:23.384+0000 7fb6dbc00700 1 -- 172.21.15.94:0/3742252804 wait complete. 2024-06-03T01:06:23.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:23 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3742252804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T01:06:23.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:23 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T01:06:23.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:23 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3742252804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T01:06:23.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:23 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T01:06:23.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:23 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3742252804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T01:06:23.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:23 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-03T01:06:24.272 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-06-03T01:06:24.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-03T01:06:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:24 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T01:06:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:24 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:06:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:24 smithi094 ceph-mon[32119]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T01:06:24.720 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:24 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T01:06:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:24 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:06:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:24 smithi107 ceph-mon[36622]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T01:06:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:24 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-03T01:06:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:24 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:06:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:24 smithi179 ceph-mon[37041]: pgmap v383: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-03T01:06:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 -- 172.21.15.94:0/3637139577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5740747b0 msgr2=0x7ff574074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 --2- 172.21.15.94:0/3637139577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5740747b0 0x7ff574074b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff568009990 tx=0x7ff568009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 -- 172.21.15.94:0/3637139577 shutdown_connections 2024-06-03T01:06:27.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 --2- 172.21.15.94:0/3637139577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff57410ec60 0x7ff574111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 --2- 172.21.15.94:0/3637139577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5740747b0 0x7ff574074b90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 --2- 172.21.15.94:0/3637139577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740750d0 0x7ff57410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 -- 172.21.15.94:0/3637139577 >> 172.21.15.94:0/3637139577 conn(0x7ff5740fdf10 msgr2=0x7ff574100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 -- 172.21.15.94:0/3637139577 shutdown_connections 2024-06-03T01:06:27.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.021+0000 7ff579e00700 1 -- 172.21.15.94:0/3637139577 wait complete. 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.022+0000 7ff579e00700 1 Processor -- start 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579e00700 1 -- start start 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740747b0 0x7ff57419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5740750d0 0x7ff57419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57410ec60 0x7ff57419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff574114900 con 0x7ff57410ec60 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff574114780 con 0x7ff5740747b0 2024-06-03T01:06:27.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff574114a80 con 0x7ff5740750d0 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff578a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740747b0 0x7ff57419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.023+0000 7ff579400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57410ec60 0x7ff57419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff573e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5740750d0 0x7ff57419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740747b0 0x7ff57419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40236/0 (socket says 172.21.15.94:40236) 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff579400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57410ec60 0x7ff57419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38292/0 (socket says 172.21.15.94:38292) 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 -- 172.21.15.94:0/3560859920 learned_addr learned my addr 172.21.15.94:0/3560859920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 -- 172.21.15.94:0/3560859920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5740750d0 msgr2=0x7ff57419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5740750d0 0x7ff57419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 -- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57410ec60 msgr2=0x7ff57419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57410ec60 0x7ff57419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 -- 172.21.15.94:0/3560859920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57419fda0 con 0x7ff5740747b0 2024-06-03T01:06:27.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.024+0000 7ff578a00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740747b0 0x7ff57419ad90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff568005910 tx=0x7ff56800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:27.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.025+0000 7ff571600700 1 -- 172.21.15.94:0/3560859920 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568027070 con 0x7ff5740747b0 2024-06-03T01:06:27.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.025+0000 7ff571600700 1 -- 172.21.15.94:0/3560859920 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff568004650 con 0x7ff5740747b0 2024-06-03T01:06:27.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.025+0000 7ff571600700 1 -- 172.21.15.94:0/3560859920 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56800f460 con 0x7ff5740747b0 2024-06-03T01:06:27.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.025+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5741a0030 con 0x7ff5740747b0 2024-06-03T01:06:27.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.025+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741a7910 con 0x7ff5740747b0 2024-06-03T01:06:27.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.026+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff574075ef0 con 0x7ff5740747b0 2024-06-03T01:06:27.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.028+0000 7ff571600700 1 -- 172.21.15.94:0/3560859920 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff56802a070 con 0x7ff5740747b0 2024-06-03T01:06:27.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.029+0000 7ff571600700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff55c0a2c20 0x7ff55c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.029+0000 7ff571600700 1 -- 172.21.15.94:0/3560859920 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff5680d06e0 con 0x7ff5740747b0 2024-06-03T01:06:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.030+0000 7ff573e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff55c0a2c20 0x7ff55c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:27.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.030+0000 7ff573e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff55c0a2c20 0x7ff55c0a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff560009680 tx=0x7ff560009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:27.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.032+0000 7ff571600700 1 -- 172.21.15.94:0/3560859920 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff568013070 con 0x7ff5740747b0 2024-06-03T01:06:27.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.285+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff57419c0f0 con 0x7ff55c0a2c20 2024-06-03T01:06:27.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.335+0000 7ff571600700 1 -- 172.21.15.94:0/3560859920 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff57419c0f0 con 0x7ff55c0a2c20 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff55c0a2c20 msgr2=0x7ff55c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff55c0a2c20 0x7ff55c0a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff560009680 tx=0x7ff560009260 comp rx=0 tx=0).stop 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740747b0 msgr2=0x7ff57419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740747b0 0x7ff57419ad90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff568005910 tx=0x7ff56800bab0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 shutdown_connections 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740747b0 0x7ff57419ad90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff55c0a2c20 0x7ff55c0a50e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff57410ec60 0x7ff57419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 --2- 172.21.15.94:0/3560859920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5740750d0 0x7ff57419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.338+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 >> 172.21.15.94:0/3560859920 conn(0x7ff5740fdf10 msgr2=0x7ff5740feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.339+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 shutdown_connections 2024-06-03T01:06:27.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:27.339+0000 7ff579e00700 1 -- 172.21.15.94:0/3560859920 wait complete. 2024-06-03T01:06:27.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:27 smithi094 ceph-mon[32119]: pgmap v384: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-03T01:06:27.600 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:06:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:06:27.300+0000 7fb5e4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:06:27.600 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:06:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:06:27.300+0000 7fb5e4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:06:27.600 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:06:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:06:27.300+0000 7fb5e4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:06:27.600 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:06:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:06:27.300+0000 7fb5e4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:06:27.600 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:06:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:06:27.300+0000 7fb5e4400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:06:27.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:27 smithi107 ceph-mon[36622]: pgmap v384: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-03T01:06:27.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:27 smithi179 ceph-mon[37041]: pgmap v384: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-06-03T01:06:28.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-03T01:06:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:28 smithi094 ceph-mon[32119]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:28 smithi094 ceph-mon[32119]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 18 op/s 2024-06-03T01:06:28.651 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:28 smithi107 ceph-mon[36622]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:28 smithi107 ceph-mon[36622]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 18 op/s 2024-06-03T01:06:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:28 smithi179 ceph-mon[37041]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:28 smithi179 ceph-mon[37041]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 18 op/s 2024-06-03T01:06:30.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:30 smithi094 ceph-mon[32119]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T01:06:30.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:30 smithi094 ceph-mon[32119]: pgmap v386: 129 pgs: 129 active+clean; 588 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-06-03T01:06:30.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:30 smithi107 ceph-mon[36622]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T01:06:30.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:30 smithi107 ceph-mon[36622]: pgmap v386: 129 pgs: 129 active+clean; 588 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-06-03T01:06:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:30 smithi179 ceph-mon[37041]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-03T01:06:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:30 smithi179 ceph-mon[37041]: pgmap v386: 129 pgs: 129 active+clean; 588 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.2 KiB/s wr, 17 op/s 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 -- 172.21.15.94:0/3897565292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff184102740 msgr2=0x7ff184102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 --2- 172.21.15.94:0/3897565292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff184102740 0x7ff184102b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff1740099c0 tx=0x7ff174009cd0 comp rx=0 tx=0).stop 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 -- 172.21.15.94:0/3897565292 shutdown_connections 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 --2- 172.21.15.94:0/3897565292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff184107840 0x7ff18410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 --2- 172.21.15.94:0/3897565292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff184102740 0x7ff184102b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 --2- 172.21.15.94:0/3897565292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1841030f0 0x7ff184107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 -- 172.21.15.94:0/3897565292 >> 172.21.15.94:0/3897565292 conn(0x7ff1840795f0 msgr2=0x7ff184079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:30.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.926+0000 7ff189800700 1 -- 172.21.15.94:0/3897565292 shutdown_connections 2024-06-03T01:06:30.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.927+0000 7ff189800700 1 -- 172.21.15.94:0/3897565292 wait complete. 2024-06-03T01:06:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.928+0000 7ff189800700 1 Processor -- start 2024-06-03T01:06:30.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff189800700 1 -- start start 2024-06-03T01:06:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff189800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff184102740 0x7ff18419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff189800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1841030f0 0x7ff18419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff189800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff184107840 0x7ff18419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff189800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff184112780 con 0x7ff1841030f0 2024-06-03T01:06:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff189800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff184112600 con 0x7ff184102740 2024-06-03T01:06:30.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff189800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff184112900 con 0x7ff184107840 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff182000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1841030f0 0x7ff18419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff182a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff184102740 0x7ff18419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff183400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff184107840 0x7ff18419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff182a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff184102740 0x7ff18419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40282/0 (socket says 172.21.15.94:40282) 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.929+0000 7ff182a00700 1 -- 172.21.15.94:0/4088291170 learned_addr learned my addr 172.21.15.94:0/4088291170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff183400700 1 -- 172.21.15.94:0/4088291170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff184102740 msgr2=0x7ff18419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff183400700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff184102740 0x7ff18419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:30.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff183400700 1 -- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1841030f0 msgr2=0x7ff18419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff183400700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1841030f0 0x7ff18419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff183400700 1 -- 172.21.15.94:0/4088291170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18419fdf0 con 0x7ff184107840 2024-06-03T01:06:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff182000700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1841030f0 0x7ff18419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:06:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff183400700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff184107840 0x7ff18419f6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00efc0 tx=0x7ff17c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:30.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.930+0000 7ff17b400700 1 -- 172.21.15.94:0/4088291170 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0092e0 con 0x7ff184107840 2024-06-03T01:06:30.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.931+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1841a0080 con 0x7ff184107840 2024-06-03T01:06:30.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.931+0000 7ff17b400700 1 -- 172.21.15.94:0/4088291170 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17c004bb0 con 0x7ff184107840 2024-06-03T01:06:30.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.931+0000 7ff17b400700 1 -- 172.21.15.94:0/4088291170 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c004280 con 0x7ff184107840 2024-06-03T01:06:30.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.931+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1841a7960 con 0x7ff184107840 2024-06-03T01:06:30.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.932+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff184069180 con 0x7ff184107840 2024-06-03T01:06:30.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.934+0000 7ff17b400700 1 -- 172.21.15.94:0/4088291170 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff17c01b030 con 0x7ff184107840 2024-06-03T01:06:30.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.935+0000 7ff17b400700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff1680a2cb0 0x7ff1680a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.935+0000 7ff17b400700 1 -- 172.21.15.94:0/4088291170 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff17c0cf090 con 0x7ff184107840 2024-06-03T01:06:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.935+0000 7ff182a00700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff1680a2cb0 0x7ff1680a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:30.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.936+0000 7ff182a00700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff1680a2cb0 0x7ff1680a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff174009750 tx=0x7ff174005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:30.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:30.940+0000 7ff17b400700 1 -- 172.21.15.94:0/4088291170 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff17c022020 con 0x7ff184107840 2024-06-03T01:06:31.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.194+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff18410e720 con 0x7ff1680a2cb0 2024-06-03T01:06:31.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.229+0000 7ff17b400700 1 -- 172.21.15.94:0/4088291170 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff18410e720 con 0x7ff1680a2cb0 2024-06-03T01:06:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff1680a2cb0 msgr2=0x7ff1680a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff1680a2cb0 0x7ff1680a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff174009750 tx=0x7ff174005960 comp rx=0 tx=0).stop 2024-06-03T01:06:31.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff184107840 msgr2=0x7ff18419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff184107840 0x7ff18419f6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00efc0 tx=0x7ff17c00c3a0 comp rx=0 tx=0).stop 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 shutdown_connections 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff184102740 0x7ff18419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff1680a2cb0 0x7ff1680a5170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff184107840 0x7ff18419f6b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 --2- 172.21.15.94:0/4088291170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1841030f0 0x7ff18419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.232+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 >> 172.21.15.94:0/4088291170 conn(0x7ff1840795f0 msgr2=0x7ff184100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:31.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.233+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 shutdown_connections 2024-06-03T01:06:31.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:31.233+0000 7ff189800700 1 -- 172.21.15.94:0/4088291170 wait complete. 2024-06-03T01:06:31.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:31 smithi094 ceph-mon[32119]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:31 smithi107 ceph-mon[36622]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:31.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:31 smithi179 ceph-mon[37041]: from='client.24709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:32.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-03T01:06:32.603 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:32 smithi179 ceph-mon[37041]: pgmap v387: 129 pgs: 129 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-06-03T01:06:32.757 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:32 smithi094 ceph-mon[32119]: pgmap v387: 129 pgs: 129 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-06-03T01:06:32.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:32 smithi107 ceph-mon[36622]: pgmap v387: 129 pgs: 129 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-06-03T01:06:34.596 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-06-03T01:06:34.596 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:34 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-06-03T01:06:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-06-03T01:06:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:34 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-06-03T01:06:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-06-03T01:06:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:34 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 -- 172.21.15.94:0/1761773212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d441046b0 msgr2=0x7f1d44104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 --2- 172.21.15.94:0/1761773212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d441046b0 0x7f1d44104b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009990 tx=0x7f1d2c009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 -- 172.21.15.94:0/1761773212 shutdown_connections 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 --2- 172.21.15.94:0/1761773212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d4410a6b0 0x7f1d4410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 --2- 172.21.15.94:0/1761773212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d44105070 0x7f1d44111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 --2- 172.21.15.94:0/1761773212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d441046b0 0x7f1d44104b30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 -- 172.21.15.94:0/1761773212 >> 172.21.15.94:0/1761773212 conn(0x7f1d441001a0 msgr2=0x7f1d441025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.915+0000 7f1d49400700 1 -- 172.21.15.94:0/1761773212 shutdown_connections 2024-06-03T01:06:34.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.916+0000 7f1d49400700 1 -- 172.21.15.94:0/1761773212 wait complete. 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 Processor -- start 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 -- start start 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d441046b0 0x7f1d4419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d44105070 0x7f1d4419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4410a6b0 0x7f1d4419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441149e0 con 0x7f1d4410a6b0 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44114860 con 0x7f1d44105070 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d49400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44114b60 con 0x7f1d441046b0 2024-06-03T01:06:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.917+0000 7f1d42800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d441046b0 0x7f1d4419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d441046b0 0x7f1d4419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:48840/0 (socket says 172.21.15.94:48840) 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 -- 172.21.15.94:0/2525827398 learned_addr learned my addr 172.21.15.94:0/2525827398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d3be00700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d44105070 0x7f1d4419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d43200700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4410a6b0 0x7f1d4419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 -- 172.21.15.94:0/2525827398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d44105070 msgr2=0x7f1d4419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d44105070 0x7f1d4419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 -- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4410a6b0 msgr2=0x7f1d4419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4410a6b0 0x7f1d4419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 -- 172.21.15.94:0/2525827398 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4419fe00 con 0x7f1d441046b0 2024-06-03T01:06:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d42800700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d441046b0 0x7f1d4419adf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e9a0 tx=0x7f1d3400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.918+0000 7f1d3b400700 1 -- 172.21.15.94:0/2525827398 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400ca10 con 0x7f1d441046b0 2024-06-03T01:06:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.919+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441a00f0 con 0x7f1d441046b0 2024-06-03T01:06:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.919+0000 7f1d3b400700 1 -- 172.21.15.94:0/2525827398 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3400cbb0 con 0x7f1d441046b0 2024-06-03T01:06:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.919+0000 7f1d3b400700 1 -- 172.21.15.94:0/2525827398 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34018680 con 0x7f1d441046b0 2024-06-03T01:06:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.919+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441a7970 con 0x7f1d441046b0 2024-06-03T01:06:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.920+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d44069180 con 0x7f1d441046b0 2024-06-03T01:06:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.922+0000 7f1d3b400700 1 -- 172.21.15.94:0/2525827398 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d34010430 con 0x7f1d441046b0 2024-06-03T01:06:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.923+0000 7f1d3b400700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1d240a2d30 0x7f1d240a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.923+0000 7f1d3b400700 1 -- 172.21.15.94:0/2525827398 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9129+0+0 (secure 0 0 0) 0x7f1d340cf6e0 con 0x7f1d441046b0 2024-06-03T01:06:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.923+0000 7f1d3be00700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1d240a2d30 0x7f1d240a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:34.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.924+0000 7f1d3be00700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1d240a2d30 0x7f1d240a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0098e0 tx=0x7f1d2c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:34.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:34.927+0000 7f1d3b400700 1 -- 172.21.15.94:0/2525827398 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d3409c100 con 0x7f1d441046b0 2024-06-03T01:06:35.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.187+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1d440628d0 con 0x7f1d240a2d30 2024-06-03T01:06:35.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.204+0000 7f1d3b400700 1 -- 172.21.15.94:0/2525827398 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1d440628d0 con 0x7f1d240a2d30 2024-06-03T01:06:35.204 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_0/454de15c-64be-4b5b-a66f-7c867950194b 2024-06-03T01:06:35.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1d240a2d30 msgr2=0x7f1d240a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:35.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1d240a2d30 0x7f1d240a51f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0098e0 tx=0x7f1d2c005960 comp rx=0 tx=0).stop 2024-06-03T01:06:35.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d441046b0 msgr2=0x7f1d4419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:35.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d441046b0 0x7f1d4419adf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e9a0 tx=0x7f1d3400ecb0 comp rx=0 tx=0).stop 2024-06-03T01:06:35.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 shutdown_connections 2024-06-03T01:06:35.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1d240a2d30 0x7f1d240a51f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:35.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4410a6b0 0x7f1d4419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:35.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1d441046b0 0x7f1d4419adf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:35.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 --2- 172.21.15.94:0/2525827398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d44105070 0x7f1d4419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:35.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.207+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 >> 172.21.15.94:0/2525827398 conn(0x7f1d441001a0 msgr2=0x7f1d44100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:35.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.208+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 shutdown_connections 2024-06-03T01:06:35.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:35.208+0000 7f1d49400700 1 -- 172.21.15.94:0/2525827398 wait complete. 2024-06-03T01:06:35.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:35 smithi094 ceph-mon[32119]: pgmap v388: 129 pgs: 129 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-06-03T01:06:35.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-06-03T01:06:35.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:35 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-06-03T01:06:35.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:35 smithi094 ceph-mon[32119]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T01:06:35.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:35 smithi094 ceph-mon[32119]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T01:06:35.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:35 smithi107 ceph-mon[36622]: pgmap v388: 129 pgs: 129 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-06-03T01:06:35.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-06-03T01:06:35.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:35 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-06-03T01:06:35.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:35 smithi107 ceph-mon[36622]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T01:06:35.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:35 smithi107 ceph-mon[36622]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T01:06:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:35 smithi179 ceph-mon[37041]: pgmap v388: 129 pgs: 129 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-06-03T01:06:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-06-03T01:06:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:35 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2024-06-03T01:06:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:35 smithi179 ceph-mon[37041]: osdmap e99: 12 total, 12 up, 12 in 2024-06-03T01:06:35.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:35 smithi179 ceph-mon[37041]: osdmap e100: 12 total, 12 up, 12 in 2024-06-03T01:06:36.139 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-03T01:06:36.583 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:36.735 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:36 smithi094 ceph-mon[32119]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:36 smithi107 ceph-mon[36622]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:36 smithi179 ceph-mon[37041]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:37 smithi094 ceph-mon[32119]: pgmap v391: 129 pgs: 1 peering, 128 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-06-03T01:06:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:37 smithi094 ceph-mon[32119]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:06:37.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:37 smithi107 ceph-mon[36622]: pgmap v391: 129 pgs: 1 peering, 128 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-06-03T01:06:37.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:37 smithi107 ceph-mon[36622]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:06:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:37 smithi179 ceph-mon[37041]: pgmap v391: 129 pgs: 1 peering, 128 active+clean; 608 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-06-03T01:06:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:37 smithi179 ceph-mon[37041]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-03T01:06:38.650 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:38 smithi094 ceph-mon[32119]: pgmap v392: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-03T01:06:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:38 smithi107 ceph-mon[36622]: pgmap v392: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-03T01:06:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:38 smithi179 ceph-mon[37041]: pgmap v392: 129 pgs: 1 peering, 128 active+clean; 614 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-06-03T01:06:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 -- 172.21.15.94:0/459689927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac810a6b0 msgr2=0x7f7ac810aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 --2- 172.21.15.94:0/459689927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac810a6b0 0x7f7ac810aa90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009990 tx=0x7f7ab8009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 -- 172.21.15.94:0/459689927 shutdown_connections 2024-06-03T01:06:38.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 --2- 172.21.15.94:0/459689927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac810a6b0 0x7f7ac810aa90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 --2- 172.21.15.94:0/459689927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac8105070 0x7f7ac8111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 --2- 172.21.15.94:0/459689927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ac81046b0 0x7f7ac8104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 -- 172.21.15.94:0/459689927 >> 172.21.15.94:0/459689927 conn(0x7f7ac81001a0 msgr2=0x7f7ac81025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 -- 172.21.15.94:0/459689927 shutdown_connections 2024-06-03T01:06:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.933+0000 7f7acea00700 1 -- 172.21.15.94:0/459689927 wait complete. 2024-06-03T01:06:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.934+0000 7f7acea00700 1 Processor -- start 2024-06-03T01:06:38.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7acea00700 1 -- start start 2024-06-03T01:06:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7acea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac81046b0 0x7f7ac819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7acea00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ac8105070 0x7f7ac819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7acea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac810a6b0 0x7f7ac819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7acea00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81149e0 con 0x7f7ac810a6b0 2024-06-03T01:06:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7acea00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8114860 con 0x7f7ac81046b0 2024-06-03T01:06:38.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7acea00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8114b60 con 0x7f7ac8105070 2024-06-03T01:06:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7ac7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac81046b0 0x7f7ac819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7ac7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac81046b0 0x7f7ac819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:46310/0 (socket says 172.21.15.94:46310) 2024-06-03T01:06:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7ac7e00700 1 -- 172.21.15.94:0/4057532188 learned_addr learned my addr 172.21.15.94:0/4057532188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:38.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.935+0000 7f7ac7400700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ac8105070 0x7f7ac819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.936+0000 7f7acca00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac810a6b0 0x7f7ac819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.936+0000 7f7ac7400700 1 -- 172.21.15.94:0/4057532188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac81046b0 msgr2=0x7f7ac819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.936+0000 7f7ac7400700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac81046b0 0x7f7ac819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.936+0000 7f7ac7400700 1 -- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac810a6b0 msgr2=0x7f7ac819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.936+0000 7f7ac7400700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac810a6b0 0x7f7ac819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.936+0000 7f7ac7400700 1 -- 172.21.15.94:0/4057532188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac819fe30 con 0x7f7ac8105070 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.936+0000 7f7acca00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac810a6b0 0x7f7ac819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:06:38.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.937+0000 7f7ac7400700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ac8105070 0x7f7ac819b3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400e9f0 tx=0x7f7ab400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.937+0000 7f7ac4c00700 1 -- 172.21.15.94:0/4057532188 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400cbd0 con 0x7f7ac8105070 2024-06-03T01:06:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.937+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac81a0120 con 0x7f7ac8105070 2024-06-03T01:06:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.937+0000 7f7ac4c00700 1 -- 172.21.15.94:0/4057532188 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab400cd70 con 0x7f7ac8105070 2024-06-03T01:06:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.937+0000 7f7ac4c00700 1 -- 172.21.15.94:0/4057532188 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4018710 con 0x7f7ac8105070 2024-06-03T01:06:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.937+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac81a79a0 con 0x7f7ac8105070 2024-06-03T01:06:38.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.939+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac810cc40 con 0x7f7ac8105070 2024-06-03T01:06:38.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.941+0000 7f7ac4c00700 1 -- 172.21.15.94:0/4057532188 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7ab4010430 con 0x7f7ac8105070 2024-06-03T01:06:38.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.943+0000 7f7ac4c00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7aac0a2cb0 0x7f7aac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:38.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.943+0000 7f7ac4c00700 1 -- 172.21.15.94:0/4057532188 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7ab40cfc40 con 0x7f7ac8105070 2024-06-03T01:06:38.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.943+0000 7f7ac7e00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7aac0a2cb0 0x7f7aac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:38.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.944+0000 7f7ac7e00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7aac0a2cb0 0x7f7aac0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8006010 tx=0x7f7ab8009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:38.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:38.944+0000 7f7ac4c00700 1 -- 172.21.15.94:0/4057532188 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7ab409c660 con 0x7f7ac8105070 2024-06-03T01:06:39.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.200+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7ac819c180 con 0x7f7aac0a2cb0 2024-06-03T01:06:39.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.233+0000 7f7ac4c00700 1 -- 172.21.15.94:0/4057532188 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7ac819c180 con 0x7f7aac0a2cb0 2024-06-03T01:06:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7aac0a2cb0 msgr2=0x7f7aac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7aac0a2cb0 0x7f7aac0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8006010 tx=0x7f7ab8009590 comp rx=0 tx=0).stop 2024-06-03T01:06:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ac8105070 msgr2=0x7f7ac819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ac8105070 0x7f7ac819b3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400e9f0 tx=0x7f7ab400ed00 comp rx=0 tx=0).stop 2024-06-03T01:06:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 shutdown_connections 2024-06-03T01:06:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac81046b0 0x7f7ac819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:39.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7aac0a2cb0 0x7f7aac0a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ac810a6b0 0x7f7ac819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 --2- 172.21.15.94:0/4057532188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ac8105070 0x7f7ac819b3b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 >> 172.21.15.94:0/4057532188 conn(0x7f7ac81001a0 msgr2=0x7f7ac81007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 shutdown_connections 2024-06-03T01:06:39.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:39.236+0000 7f7acea00700 1 -- 172.21.15.94:0/4057532188 wait complete. 2024-06-03T01:06:39.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:39 smithi094 ceph-mon[32119]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:39.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:39 smithi107 ceph-mon[36622]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:39.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:39 smithi179 ceph-mon[37041]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:40.145 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-03T01:06:40.579 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:40.728 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:40 smithi094 ceph-mon[32119]: pgmap v393: 129 pgs: 129 active+clean; 614 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-03T01:06:40.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:40 smithi107 ceph-mon[36622]: pgmap v393: 129 pgs: 129 active+clean; 614 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-03T01:06:40.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:40 smithi179 ceph-mon[37041]: pgmap v393: 129 pgs: 129 active+clean; 614 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-03T01:06:41.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:41 smithi094 ceph-mon[32119]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:06:41.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:41 smithi094 ceph-mon[32119]: Cluster is now healthy 2024-06-03T01:06:41.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:41 smithi107 ceph-mon[36622]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:06:41.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:41 smithi107 ceph-mon[36622]: Cluster is now healthy 2024-06-03T01:06:41.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:41 smithi179 ceph-mon[37041]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-03T01:06:41.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:41 smithi179 ceph-mon[37041]: Cluster is now healthy 2024-06-03T01:06:42.510 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:42 smithi094 ceph-mon[32119]: pgmap v394: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-03T01:06:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:42 smithi107 ceph-mon[36622]: pgmap v394: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-03T01:06:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 -- 172.21.15.94:0/2569876795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 msgr2=0x7fe1e4112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2569876795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 0x7fe1e4112de0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00098f0 tx=0x7fe1d0009c00 comp rx=0 tx=0).stop 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 -- 172.21.15.94:0/2569876795 shutdown_connections 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2569876795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e40769c0 0x7fe1e4076da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2569876795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 0x7fe1e4115710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2569876795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 0x7fe1e4112de0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 -- 172.21.15.94:0/2569876795 >> 172.21.15.94:0/2569876795 conn(0x7fe1e4100170 msgr2=0x7fe1e4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 -- 172.21.15.94:0/2569876795 shutdown_connections 2024-06-03T01:06:42.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.802+0000 7fe1ec400700 1 -- 172.21.15.94:0/2569876795 wait complete. 2024-06-03T01:06:42.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.803+0000 7fe1ec400700 1 Processor -- start 2024-06-03T01:06:42.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ec400700 1 -- start start 2024-06-03T01:06:42.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ec400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e40769c0 0x7fe1e419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:42.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ec400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 0x7fe1e419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:42.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ec400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 0x7fe1e41a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:42.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ec400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4118e00 con 0x7fe1e40772e0 2024-06-03T01:06:42.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ec400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4118c80 con 0x7fe1e40769c0 2024-06-03T01:06:42.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ec400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e4118f80 con 0x7fe1e4113320 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1e9000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 0x7fe1e419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ea400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 0x7fe1e41a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1e9a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e40769c0 0x7fe1e419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1e9000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 0x7fe1e419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54786/0 (socket says 172.21.15.94:54786) 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1ea400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 0x7fe1e41a3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:48888/0 (socket says 172.21.15.94:48888) 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.804+0000 7fe1e9000700 1 -- 172.21.15.94:0/2069074340 learned_addr learned my addr 172.21.15.94:0/2069074340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1ea400700 1 -- 172.21.15.94:0/2069074340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e40769c0 msgr2=0x7fe1e419f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1ea400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e40769c0 0x7fe1e419f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1ea400700 1 -- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 msgr2=0x7fe1e419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1ea400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 0x7fe1e419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1ea400700 1 -- 172.21.15.94:0/2069074340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e41a42b0 con 0x7fe1e4113320 2024-06-03T01:06:42.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1ea400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 0x7fe1e41a3b70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe1e407acb0 tx=0x7fe1dc00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:42.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1e2000700 1 -- 172.21.15.94:0/2069074340 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0092e0 con 0x7fe1e4113320 2024-06-03T01:06:42.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.805+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e41a4540 con 0x7fe1e4113320 2024-06-03T01:06:42.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.806+0000 7fe1e2000700 1 -- 172.21.15.94:0/2069074340 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1dc004bb0 con 0x7fe1e4113320 2024-06-03T01:06:42.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.806+0000 7fe1e2000700 1 -- 172.21.15.94:0/2069074340 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc004020 con 0x7fe1e4113320 2024-06-03T01:06:42.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.806+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e41abe20 con 0x7fe1e4113320 2024-06-03T01:06:42.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.807+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e4069180 con 0x7fe1e4113320 2024-06-03T01:06:42.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.809+0000 7fe1e2000700 1 -- 172.21.15.94:0/2069074340 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe1dc01b030 con 0x7fe1e4113320 2024-06-03T01:06:42.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.810+0000 7fe1e2000700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe1cc0a2cb0 0x7fe1cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:42.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.810+0000 7fe1e2000700 1 -- 172.21.15.94:0/2069074340 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe1dc0cef00 con 0x7fe1e4113320 2024-06-03T01:06:42.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.810+0000 7fe1e9a00700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe1cc0a2cb0 0x7fe1cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:42.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.811+0000 7fe1e9a00700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe1cc0a2cb0 0x7fe1cc0a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009680 tx=0x7fe1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:42.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:42.814+0000 7fe1e2000700 1 -- 172.21.15.94:0/2069074340 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe1dc022020 con 0x7fe1e4113320 2024-06-03T01:06:42.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:42 smithi179 ceph-mon[37041]: pgmap v394: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-06-03T01:06:43.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.076+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe1e40628d0 con 0x7fe1cc0a2cb0 2024-06-03T01:06:43.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.086+0000 7fe1e2000700 1 -- 172.21.15.94:0/2069074340 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe1e40628d0 con 0x7fe1cc0a2cb0 2024-06-03T01:06:43.087 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_1/57911953-237f-46f0-b619-4974b41e3cd6 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe1cc0a2cb0 msgr2=0x7fe1cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe1cc0a2cb0 0x7fe1cc0a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009680 tx=0x7fe1d8009260 comp rx=0 tx=0).stop 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 msgr2=0x7fe1e41a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 0x7fe1e41a3b70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe1e407acb0 tx=0x7fe1dc00ef90 comp rx=0 tx=0).stop 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 shutdown_connections 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1e40769c0 0x7fe1e419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe1cc0a2cb0 0x7fe1cc0a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1e4113320 0x7fe1e41a3b70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 --2- 172.21.15.94:0/2069074340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1e40772e0 0x7fe1e419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 >> 172.21.15.94:0/2069074340 conn(0x7fe1e4100170 msgr2=0x7fe1e4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:43.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 shutdown_connections 2024-06-03T01:06:43.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:43.089+0000 7fe1ec400700 1 -- 172.21.15.94:0/2069074340 wait complete. 2024-06-03T01:06:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:43 smithi107 ceph-mon[36622]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:43 smithi094 ceph-mon[32119]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:43.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:43 smithi179 ceph-mon[37041]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:06:43.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:06:44.391 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:44.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:44 smithi094 ceph-mon[32119]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-06-03T01:06:44.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:44 smithi107 ceph-mon[36622]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-06-03T01:06:44.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:44 smithi179 ceph-mon[37041]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-06-03T01:06:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:46 smithi094 ceph-mon[32119]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-06-03T01:06:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.624+0000 7f6229400700 1 -- 172.21.15.94:0/892972474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62241046d0 msgr2=0x7f6224104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.624+0000 7f6229400700 1 --2- 172.21.15.94:0/892972474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62241046d0 0x7f6224104b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6210009990 tx=0x7f6210009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.625+0000 7f6229400700 1 -- 172.21.15.94:0/892972474 shutdown_connections 2024-06-03T01:06:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.625+0000 7f6229400700 1 --2- 172.21.15.94:0/892972474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622410a6d0 0x7f622410aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.625+0000 7f6229400700 1 --2- 172.21.15.94:0/892972474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6224105090 0x7f62241115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.625+0000 7f6229400700 1 --2- 172.21.15.94:0/892972474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62241046d0 0x7f6224104b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.625+0000 7f6229400700 1 -- 172.21.15.94:0/892972474 >> 172.21.15.94:0/892972474 conn(0x7f62241001c0 msgr2=0x7f62241025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.625+0000 7f6229400700 1 -- 172.21.15.94:0/892972474 shutdown_connections 2024-06-03T01:06:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.625+0000 7f6229400700 1 -- 172.21.15.94:0/892972474 wait complete. 2024-06-03T01:06:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.626+0000 7f6229400700 1 Processor -- start 2024-06-03T01:06:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.626+0000 7f6229400700 1 -- start start 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6229400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62241046d0 0x7f622419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6229400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6224105090 0x7f622419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6229400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622410a6d0 0x7f622419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6229400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6224114ab0 con 0x7f62241046d0 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6229400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6224114930 con 0x7f6224105090 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6229400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6224114c30 con 0x7f622410a6d0 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6222800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62241046d0 0x7f622419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6221e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6224105090 0x7f622419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6221e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6224105090 0x7f622419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37446/0 (socket says 172.21.15.94:37446) 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6221e00700 1 -- 172.21.15.94:0/2469690649 learned_addr learned my addr 172.21.15.94:0/2469690649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.627+0000 7f6223200700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622410a6d0 0x7f622419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f6221e00700 1 -- 172.21.15.94:0/2469690649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622410a6d0 msgr2=0x7f622419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f6221e00700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622410a6d0 0x7f622419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f6221e00700 1 -- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62241046d0 msgr2=0x7f622419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f6221e00700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62241046d0 0x7f622419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f6221e00700 1 -- 172.21.15.94:0/2469690649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622419fe60 con 0x7f6224105090 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f6221e00700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6224105090 0x7f622419b3e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6210000c00 tx=0x7f621000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f621b400700 1 -- 172.21.15.94:0/2469690649 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210027070 con 0x7f6224105090 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.628+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62241a00f0 con 0x7f6224105090 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.629+0000 7f621b400700 1 -- 172.21.15.94:0/2469690649 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62100056d0 con 0x7f6224105090 2024-06-03T01:06:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.629+0000 7f621b400700 1 -- 172.21.15.94:0/2469690649 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210022e10 con 0x7f6224105090 2024-06-03T01:06:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.629+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241a79d0 con 0x7f6224105090 2024-06-03T01:06:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.630+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6224069180 con 0x7f6224105090 2024-06-03T01:06:46.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.632+0000 7f621b400700 1 -- 172.21.15.94:0/2469690649 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f621002a070 con 0x7f6224105090 2024-06-03T01:06:46.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.633+0000 7f621b400700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f62080a2d30 0x7f62080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:46.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.633+0000 7f621b400700 1 -- 172.21.15.94:0/2469690649 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f62100d0b70 con 0x7f6224105090 2024-06-03T01:06:46.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.634+0000 7f6222800700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f62080a2d30 0x7f62080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:46.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.634+0000 7f6222800700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f62080a2d30 0x7f62080a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6214009be0 tx=0x7f6214009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:46.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.635+0000 7f621b400700 1 -- 172.21.15.94:0/2469690649 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6210013070 con 0x7f6224105090 2024-06-03T01:06:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:46 smithi107 ceph-mon[36622]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-06-03T01:06:46.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:46 smithi179 ceph-mon[37041]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-06-03T01:06:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.904+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f622402dca0 con 0x7f6224105090 2024-06-03T01:06:46.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.906+0000 7f621b400700 1 -- 172.21.15.94:0/2469690649 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f621009d620 con 0x7f6224105090 2024-06-03T01:06:46.907 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:06:46.907 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:06:22.586009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24478,24485,24490,24505,24524]},"id":1}]} 2024-06-03T01:06:46.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f62080a2d30 msgr2=0x7f62080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:46.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f62080a2d30 0x7f62080a51f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6214009be0 tx=0x7f6214009260 comp rx=0 tx=0).stop 2024-06-03T01:06:46.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6224105090 msgr2=0x7f622419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:46.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6224105090 0x7f622419b3e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6210000c00 tx=0x7f621000bab0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 shutdown_connections 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f62080a2d30 0x7f62080a51f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f622410a6d0 0x7f622419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62241046d0 0x7f622419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 --2- 172.21.15.94:0/2469690649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6224105090 0x7f622419b3e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 >> 172.21.15.94:0/2469690649 conn(0x7f62241001c0 msgr2=0x7f62241006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 shutdown_connections 2024-06-03T01:06:46.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:46.909+0000 7f6229400700 1 -- 172.21.15.94:0/2469690649 wait complete. 2024-06-03T01:06:46.912 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-06-03T01:06:47.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:47 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2469690649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:06:47.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:47 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2469690649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:06:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:47 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2469690649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:06:47.900 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:06:48.335 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:06:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:48 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:48 smithi179 ceph-mon[37041]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T01:06:48.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:06:48.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:48.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:48 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:48.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:48 smithi094 ceph-mon[32119]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T01:06:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:06:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:48 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:48 smithi107 ceph-mon[36622]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-06-03T01:06:50.297 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.297 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.297 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.297 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:50 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:50 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:50 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.623+0000 7faf18400700 1 -- 172.21.15.94:0/3515175644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10107840 msgr2=0x7faf1010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.623+0000 7faf18400700 1 --2- 172.21.15.94:0/3515175644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10107840 0x7faf1010f3c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b0d0 tx=0x7faf0800b3e0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.624+0000 7faf18400700 1 -- 172.21.15.94:0/3515175644 shutdown_connections 2024-06-03T01:06:50.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.624+0000 7faf18400700 1 --2- 172.21.15.94:0/3515175644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf10102740 0x7faf10102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.624+0000 7faf18400700 1 --2- 172.21.15.94:0/3515175644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10107840 0x7faf1010f3c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.624+0000 7faf18400700 1 --2- 172.21.15.94:0/3515175644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 0x7faf10107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.624+0000 7faf18400700 1 -- 172.21.15.94:0/3515175644 >> 172.21.15.94:0/3515175644 conn(0x7faf100795f0 msgr2=0x7faf10079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.624+0000 7faf18400700 1 -- 172.21.15.94:0/3515175644 shutdown_connections 2024-06-03T01:06:50.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.624+0000 7faf18400700 1 -- 172.21.15.94:0/3515175644 wait complete. 2024-06-03T01:06:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.625+0000 7faf18400700 1 Processor -- start 2024-06-03T01:06:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.625+0000 7faf18400700 1 -- start start 2024-06-03T01:06:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.625+0000 7faf18400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10102740 0x7faf1019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.625+0000 7faf18400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 0x7faf1019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf18400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf10107840 0x7faf1019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf18400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf10112780 con 0x7faf10102740 2024-06-03T01:06:50.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf18400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf10112600 con 0x7faf10107840 2024-06-03T01:06:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf18400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf10112900 con 0x7faf101030f0 2024-06-03T01:06:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10102740 0x7faf1019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 0x7faf1019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf16400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf10107840 0x7faf1019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10102740 0x7faf1019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60052/0 (socket says 172.21.15.94:60052) 2024-06-03T01:06:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 0x7faf1019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51076/0 (socket says 172.21.15.94:51076) 2024-06-03T01:06:50.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15a00700 1 -- 172.21.15.94:0/2076763290 learned_addr learned my addr 172.21.15.94:0/2076763290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15000700 1 -- 172.21.15.94:0/2076763290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf10107840 msgr2=0x7faf1019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15000700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf10107840 0x7faf1019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15000700 1 -- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10102740 msgr2=0x7faf1019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15000700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10102740 0x7faf1019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15000700 1 -- 172.21.15.94:0/2076763290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1019fd20 con 0x7faf101030f0 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.626+0000 7faf15a00700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10102740 0x7faf1019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.627+0000 7faf15000700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 0x7faf1019b250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faefc00e9f0 tx=0x7faefc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.627+0000 7faf0e000700 1 -- 172.21.15.94:0/2076763290 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc004220 con 0x7faf101030f0 2024-06-03T01:06:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.627+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf101a0010 con 0x7faf101030f0 2024-06-03T01:06:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.627+0000 7faf0e000700 1 -- 172.21.15.94:0/2076763290 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faefc004d10 con 0x7faf101030f0 2024-06-03T01:06:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.627+0000 7faf0e000700 1 -- 172.21.15.94:0/2076763290 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc005020 con 0x7faf101030f0 2024-06-03T01:06:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.627+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf101a7890 con 0x7faf101030f0 2024-06-03T01:06:50.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.628+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf10069180 con 0x7faf101030f0 2024-06-03T01:06:50.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.631+0000 7faf0e000700 1 -- 172.21.15.94:0/2076763290 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faefc042b10 con 0x7faf101030f0 2024-06-03T01:06:50.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.632+0000 7faf0e000700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7faef80a2cb0 0x7faef80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:50.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.632+0000 7faf0e000700 1 -- 172.21.15.94:0/2076763290 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7faefc0cf400 con 0x7faf101030f0 2024-06-03T01:06:50.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.632+0000 7faf15a00700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7faef80a2cb0 0x7faef80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:50.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.633+0000 7faf15a00700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7faef80a2cb0 0x7faef80a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf04011550 tx=0x7faf04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.634+0000 7faf0e000700 1 -- 172.21.15.94:0/2076763290 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faefc01f3a0 con 0x7faf101030f0 2024-06-03T01:06:50.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.870+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf1019c070 con 0x7faf101030f0 2024-06-03T01:06:50.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.872+0000 7faf0e000700 1 -- 172.21.15.94:0/2076763290 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24723 (secure 0 0 0) 0x7faf1019c070 con 0x7faf101030f0 2024-06-03T01:06:50.873 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:06:50.873 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":100,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:06:35.341979+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.875+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7faef80a2cb0 msgr2=0x7faef80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.875+0000 7faf18400700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7faef80a2cb0 0x7faef80a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf04011550 tx=0x7faf04009260 comp rx=0 tx=0).stop 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.875+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 msgr2=0x7faf1019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.875+0000 7faf18400700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 0x7faf1019b250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faefc00e9f0 tx=0x7faefc00ed00 comp rx=0 tx=0).stop 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 shutdown_connections 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf10107840 0x7faf1019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7faef80a2cb0 0x7faef80a5170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf10102740 0x7faf1019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 --2- 172.21.15.94:0/2076763290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf101030f0 0x7faf1019b250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:50.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 >> 172.21.15.94:0/2076763290 conn(0x7faf100795f0 msgr2=0x7faf101006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:50.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 shutdown_connections 2024-06-03T01:06:50.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:50.876+0000 7faf18400700 1 -- 172.21.15.94:0/2076763290 wait complete. 2024-06-03T01:06:51.436 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:51 smithi094 ceph-mon[32119]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-03T01:06:51.436 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:51 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2076763290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:51.506 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:51 smithi179 ceph-mon[37041]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-03T01:06:51.506 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:51 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2076763290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:51.532 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:51 smithi107 ceph-mon[36622]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-06-03T01:06:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:51 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2076763290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:06:52.200 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-03T01:06:52.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T01:06:52.647 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:52 smithi107 ceph-mon[36622]: pgmap v399: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-06-03T01:06:52.800 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:52 smithi094 ceph-mon[32119]: pgmap v399: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-06-03T01:06:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:52 smithi179 ceph-mon[37041]: pgmap v399: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc24ca00700 1 -- 172.21.15.94:0/3820372972 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00bc40 con 0x7fc250074fe0 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc254e00700 1 -- 172.21.15.94:0/3820372972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc250074fe0 msgr2=0x7fc25010e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc254e00700 1 --2- 172.21.15.94:0/3820372972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc250074fe0 0x7fc25010e950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009990 tx=0x7fc23c009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc254e00700 1 -- 172.21.15.94:0/3820372972 shutdown_connections 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc254e00700 1 --2- 172.21.15.94:0/3820372972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 0x7fc250074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc254e00700 1 --2- 172.21.15.94:0/3820372972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc25010ee90 0x7fc250111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc254e00700 1 --2- 172.21.15.94:0/3820372972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc250074fe0 0x7fc25010e950 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:54.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.883+0000 7fc254e00700 1 -- 172.21.15.94:0/3820372972 >> 172.21.15.94:0/3820372972 conn(0x7fc2500795f0 msgr2=0x7fc250079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:54.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.884+0000 7fc254e00700 1 -- 172.21.15.94:0/3820372972 shutdown_connections 2024-06-03T01:06:54.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.884+0000 7fc254e00700 1 -- 172.21.15.94:0/3820372972 wait complete. 2024-06-03T01:06:54.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 Processor -- start 2024-06-03T01:06:54.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 -- start start 2024-06-03T01:06:54.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 0x7fc25019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:54.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc250074fe0 0x7fc25019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:54.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc25010ee90 0x7fc25019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:54.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc2501149a0 con 0x7fc25010ee90 2024-06-03T01:06:54.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc250114820 con 0x7fc2500746c0 2024-06-03T01:06:54.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.885+0000 7fc254e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc250114b20 con 0x7fc250074fe0 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc25010ee90 0x7fc25019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 0x7fc25019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 0x7fc25019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51114/0 (socket says 172.21.15.94:51114) 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 -- 172.21.15.94:0/2365663941 learned_addr learned my addr 172.21.15.94:0/2365663941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24d400700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc250074fe0 0x7fc25019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 -- 172.21.15.94:0/2365663941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc250074fe0 msgr2=0x7fc25019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc250074fe0 0x7fc25019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 -- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc25010ee90 msgr2=0x7fc25019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc25010ee90 0x7fc25019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.886+0000 7fc24de00700 1 -- 172.21.15.94:0/2365663941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25019fe10 con 0x7fc2500746c0 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.887+0000 7fc24de00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 0x7fc25019ae50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc24000eb30 tx=0x7fc24000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:54.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.887+0000 7fc246a00700 1 -- 172.21.15.94:0/2365663941 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24000cbd0 con 0x7fc2500746c0 2024-06-03T01:06:54.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.887+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2501a0100 con 0x7fc2500746c0 2024-06-03T01:06:54.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.887+0000 7fc246a00700 1 -- 172.21.15.94:0/2365663941 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240004500 con 0x7fc2500746c0 2024-06-03T01:06:54.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.887+0000 7fc246a00700 1 -- 172.21.15.94:0/2365663941 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240010430 con 0x7fc2500746c0 2024-06-03T01:06:54.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.887+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501a7980 con 0x7fc2500746c0 2024-06-03T01:06:54.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.888+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc250069180 con 0x7fc2500746c0 2024-06-03T01:06:54.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.890+0000 7fc246a00700 1 -- 172.21.15.94:0/2365663941 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc240003f00 con 0x7fc2500746c0 2024-06-03T01:06:54.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.892+0000 7fc246a00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc2340a2cb0 0x7fc2340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:54.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.892+0000 7fc246a00700 1 -- 172.21.15.94:0/2365663941 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc2400cf910 con 0x7fc2500746c0 2024-06-03T01:06:54.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.892+0000 7fc24d400700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc2340a2cb0 0x7fc2340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:54.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.893+0000 7fc24d400700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc2340a2cb0 0x7fc2340a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b2e0 tx=0x7fc23c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:54.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:54.896+0000 7fc246a00700 1 -- 172.21.15.94:0/2365663941 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc24009c330 con 0x7fc2500746c0 2024-06-03T01:06:55.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.118+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2500628d0 con 0x7fc2340a2cb0 2024-06-03T01:06:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.131+0000 7fc246a00700 1 -- 172.21.15.94:0/2365663941 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340535 (secure 0 0 0) 0x7fc23c079250 con 0x7fc2340a2cb0 2024-06-03T01:06:55.139 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.138+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc2340a2cb0 msgr2=0x7fc2340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.138+0000 7fc254e00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc2340a2cb0 0x7fc2340a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b2e0 tx=0x7fc23c005960 comp rx=0 tx=0).stop 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.138+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 msgr2=0x7fc25019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.138+0000 7fc254e00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 0x7fc25019ae50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc24000eb30 tx=0x7fc24000ee40 comp rx=0 tx=0).stop 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 shutdown_connections 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2500746c0 0x7fc25019ae50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc2340a2cb0 0x7fc2340a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc25010ee90 0x7fc25019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 --2- 172.21.15.94:0/2365663941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc250074fe0 0x7fc25019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 >> 172.21.15.94:0/2365663941 conn(0x7fc2500795f0 msgr2=0x7fc2500ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:55.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 shutdown_connections 2024-06-03T01:06:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:55.139+0000 7fc254e00700 1 -- 172.21.15.94:0/2365663941 wait complete. 2024-06-03T01:06:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-06-03T01:06:55.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:55.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:55.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:55 smithi094 ceph-mon[32119]: pgmap v400: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-03T01:06:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:55.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:55 smithi179 ceph-mon[37041]: pgmap v400: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-03T01:06:55.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:55.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:55.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:55 smithi107 ceph-mon[36622]: pgmap v400: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-03T01:06:56.009 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-06-03T01:06:54.310967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665638,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":508,"num_read_kb":387,"num_write":436,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402348,"kb_used_data":15740,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539476,"statfs":{"total":1151940427776,"available":1151528423424,"internally_reserved":0,"allocated":16117760,"data_stored":8863736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19581,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":70,"num_write":15,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003974"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355301+0000","last_change":"2024-06-03T01:04:54.382830+0000","last_active":"2024-06-03T01:06:35.355301+0000","last_peered":"2024-06-03T01:06:35.355301+0000","last_clean":"2024-06-03T01:06:35.355301+0000","last_became_active":"2024-06-03T01:04:54.382328+0000","last_became_peered":"2024-06-03T01:04:54.382328+0000","last_unstale":"2024-06-03T01:06:35.355301+0000","last_undegraded":"2024-06-03T01:06:35.355301+0000","last_fullsized":"2024-06-03T01:06:35.355301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:59.037188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597232+0000","last_change":"2024-06-03T01:04:54.380179+0000","last_active":"2024-06-03T01:06:35.597232+0000","last_peered":"2024-06-03T01:06:35.597232+0000","last_clean":"2024-06-03T01:06:35.597232+0000","last_became_active":"2024-06-03T01:04:54.370975+0000","last_became_peered":"2024-06-03T01:04:54.370975+0000","last_unstale":"2024-06-03T01:06:35.597232+0000","last_undegraded":"2024-06-03T01:06:35.597232+0000","last_fullsized":"2024-06-03T01:06:35.597232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:41.133053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596810+0000","last_change":"2024-06-03T01:04:54.381423+0000","last_active":"2024-06-03T01:06:35.596810+0000","last_peered":"2024-06-03T01:06:35.596810+0000","last_clean":"2024-06-03T01:06:35.596810+0000","last_became_active":"2024-06-03T01:04:54.375659+0000","last_became_peered":"2024-06-03T01:04:54.375659+0000","last_unstale":"2024-06-03T01:06:35.596810+0000","last_undegraded":"2024-06-03T01:06:35.596810+0000","last_fullsized":"2024-06-03T01:06:35.596810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:50.294425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355121+0000","last_change":"2024-06-03T01:04:54.372494+0000","last_active":"2024-06-03T01:06:35.355121+0000","last_peered":"2024-06-03T01:06:35.355121+0000","last_clean":"2024-06-03T01:06:35.355121+0000","last_became_active":"2024-06-03T01:04:54.371744+0000","last_became_peered":"2024-06-03T01:04:54.371744+0000","last_unstale":"2024-06-03T01:06:35.355121+0000","last_undegraded":"2024-06-03T01:06:35.355121+0000","last_fullsized":"2024-06-03T01:06:35.355121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:01.887751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837420+0000","last_change":"2024-06-03T01:04:54.394675+0000","last_active":"2024-06-03T01:06:35.837420+0000","last_peered":"2024-06-03T01:06:35.837420+0000","last_clean":"2024-06-03T01:06:35.837420+0000","last_became_active":"2024-06-03T01:04:54.394246+0000","last_became_peered":"2024-06-03T01:04:54.394246+0000","last_unstale":"2024-06-03T01:06:35.837420+0000","last_undegraded":"2024-06-03T01:06:35.837420+0000","last_fullsized":"2024-06-03T01:06:35.837420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:48:47.331598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363884+0000","last_change":"2024-06-03T01:04:54.378765+0000","last_active":"2024-06-03T01:06:35.363884+0000","last_peered":"2024-06-03T01:06:35.363884+0000","last_clean":"2024-06-03T01:06:35.363884+0000","last_became_active":"2024-06-03T01:04:54.378306+0000","last_became_peered":"2024-06-03T01:04:54.378306+0000","last_unstale":"2024-06-03T01:06:35.363884+0000","last_undegraded":"2024-06-03T01:06:35.363884+0000","last_fullsized":"2024-06-03T01:06:35.363884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:26:20.321196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382222+0000","last_change":"2024-06-03T01:05:35.252114+0000","last_active":"2024-06-03T01:06:35.382222+0000","last_peered":"2024-06-03T01:06:35.382222+0000","last_clean":"2024-06-03T01:06:35.382222+0000","last_became_active":"2024-06-03T01:05:35.251849+0000","last_became_peered":"2024-06-03T01:05:35.251849+0000","last_unstale":"2024-06-03T01:06:35.382222+0000","last_undegraded":"2024-06-03T01:06:35.382222+0000","last_fullsized":"2024-06-03T01:06:35.382222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:18.942723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840012+0000","last_change":"2024-06-03T01:04:54.397423+0000","last_active":"2024-06-03T01:06:35.840012+0000","last_peered":"2024-06-03T01:06:35.840012+0000","last_clean":"2024-06-03T01:06:35.840012+0000","last_became_active":"2024-06-03T01:04:54.394329+0000","last_became_peered":"2024-06-03T01:04:54.394329+0000","last_unstale":"2024-06-03T01:06:35.840012+0000","last_undegraded":"2024-06-03T01:06:35.840012+0000","last_fullsized":"2024-06-03T01:06:35.840012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:04.020364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838582+0000","last_change":"2024-06-03T01:04:54.368589+0000","last_active":"2024-06-03T01:06:35.838582+0000","last_peered":"2024-06-03T01:06:35.838582+0000","last_clean":"2024-06-03T01:06:35.838582+0000","last_became_active":"2024-06-03T01:04:54.368068+0000","last_became_peered":"2024-06-03T01:04:54.368068+0000","last_unstale":"2024-06-03T01:06:35.838582+0000","last_undegraded":"2024-06-03T01:06:35.838582+0000","last_fullsized":"2024-06-03T01:06:35.838582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:59:44.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.364502+0000","last_change":"2024-06-03T01:04:54.367488+0000","last_active":"2024-06-03T01:06:35.364502+0000","last_peered":"2024-06-03T01:06:35.364502+0000","last_clean":"2024-06-03T01:06:35.364502+0000","last_became_active":"2024-06-03T01:04:54.367022+0000","last_became_peered":"2024-06-03T01:04:54.367022+0000","last_unstale":"2024-06-03T01:06:35.364502+0000","last_undegraded":"2024-06-03T01:06:35.364502+0000","last_fullsized":"2024-06-03T01:06:35.364502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:52:27.697948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382593+0000","last_change":"2024-06-03T01:04:54.366790+0000","last_active":"2024-06-03T01:06:35.382593+0000","last_peered":"2024-06-03T01:06:35.382593+0000","last_clean":"2024-06-03T01:06:35.382593+0000","last_became_active":"2024-06-03T01:04:54.366456+0000","last_became_peered":"2024-06-03T01:04:54.366456+0000","last_unstale":"2024-06-03T01:06:35.382593+0000","last_undegraded":"2024-06-03T01:06:35.382593+0000","last_fullsized":"2024-06-03T01:06:35.382593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:15:42.495284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355088+0000","last_change":"2024-06-03T01:04:54.369483+0000","last_active":"2024-06-03T01:06:35.355088+0000","last_peered":"2024-06-03T01:06:35.355088+0000","last_clean":"2024-06-03T01:06:35.355088+0000","last_became_active":"2024-06-03T01:04:54.369118+0000","last_became_peered":"2024-06-03T01:04:54.369118+0000","last_unstale":"2024-06-03T01:06:35.355088+0000","last_undegraded":"2024-06-03T01:06:35.355088+0000","last_fullsized":"2024-06-03T01:06:35.355088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:18.218426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350165+0000","last_change":"2024-06-03T01:04:54.380648+0000","last_active":"2024-06-03T01:06:35.350165+0000","last_peered":"2024-06-03T01:06:35.350165+0000","last_clean":"2024-06-03T01:06:35.350165+0000","last_became_active":"2024-06-03T01:04:54.380195+0000","last_became_peered":"2024-06-03T01:04:54.380195+0000","last_unstale":"2024-06-03T01:06:35.350165+0000","last_undegraded":"2024-06-03T01:06:35.350165+0000","last_fullsized":"2024-06-03T01:06:35.350165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:04.991858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596367+0000","last_change":"2024-06-03T01:04:54.367250+0000","last_active":"2024-06-03T01:06:35.596367+0000","last_peered":"2024-06-03T01:06:35.596367+0000","last_clean":"2024-06-03T01:06:35.596367+0000","last_became_active":"2024-06-03T01:04:54.364817+0000","last_became_peered":"2024-06-03T01:04:54.364817+0000","last_unstale":"2024-06-03T01:06:35.596367+0000","last_undegraded":"2024-06-03T01:06:35.596367+0000","last_fullsized":"2024-06-03T01:06:35.596367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:55:20.948692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596342+0000","last_change":"2024-06-03T01:04:54.367640+0000","last_active":"2024-06-03T01:06:35.596342+0000","last_peered":"2024-06-03T01:06:35.596342+0000","last_clean":"2024-06-03T01:06:35.596342+0000","last_became_active":"2024-06-03T01:04:54.367080+0000","last_became_peered":"2024-06-03T01:04:54.367080+0000","last_unstale":"2024-06-03T01:06:35.596342+0000","last_undegraded":"2024-06-03T01:06:35.596342+0000","last_fullsized":"2024-06-03T01:06:35.596342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:43:23.469092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837165+0000","last_change":"2024-06-03T01:04:54.376236+0000","last_active":"2024-06-03T01:06:35.837165+0000","last_peered":"2024-06-03T01:06:35.837165+0000","last_clean":"2024-06-03T01:06:35.837165+0000","last_became_active":"2024-06-03T01:04:54.375866+0000","last_became_peered":"2024-06-03T01:04:54.375866+0000","last_unstale":"2024-06-03T01:06:35.837165+0000","last_undegraded":"2024-06-03T01:06:35.837165+0000","last_fullsized":"2024-06-03T01:06:35.837165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:45.926391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355169+0000","last_change":"2024-06-03T01:04:54.388824+0000","last_active":"2024-06-03T01:06:35.355169+0000","last_peered":"2024-06-03T01:06:35.355169+0000","last_clean":"2024-06-03T01:06:35.355169+0000","last_became_active":"2024-06-03T01:04:54.388491+0000","last_became_peered":"2024-06-03T01:04:54.388491+0000","last_unstale":"2024-06-03T01:06:35.355169+0000","last_undegraded":"2024-06-03T01:06:35.355169+0000","last_fullsized":"2024-06-03T01:06:35.355169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:48.593126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.353313+0000","last_change":"2024-06-03T01:04:54.377623+0000","last_active":"2024-06-03T01:06:35.353313+0000","last_peered":"2024-06-03T01:06:35.353313+0000","last_clean":"2024-06-03T01:06:35.353313+0000","last_became_active":"2024-06-03T01:04:54.377037+0000","last_became_peered":"2024-06-03T01:04:54.377037+0000","last_unstale":"2024-06-03T01:06:35.353313+0000","last_undegraded":"2024-06-03T01:06:35.353313+0000","last_fullsized":"2024-06-03T01:06:35.353313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:52:12.729604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.348167+0000","last_change":"2024-06-03T01:04:54.396946+0000","last_active":"2024-06-03T01:06:35.348167+0000","last_peered":"2024-06-03T01:06:35.348167+0000","last_clean":"2024-06-03T01:06:35.348167+0000","last_became_active":"2024-06-03T01:04:54.396695+0000","last_became_peered":"2024-06-03T01:04:54.396695+0000","last_unstale":"2024-06-03T01:06:35.348167+0000","last_undegraded":"2024-06-03T01:06:35.348167+0000","last_fullsized":"2024-06-03T01:06:35.348167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:10:57.842403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347435+0000","last_change":"2024-06-03T01:05:08.399118+0000","last_active":"2024-06-03T01:06:35.347435+0000","last_peered":"2024-06-03T01:06:35.347435+0000","last_clean":"2024-06-03T01:06:35.347435+0000","last_became_active":"2024-06-03T01:05:08.398716+0000","last_became_peered":"2024-06-03T01:05:08.398716+0000","last_unstale":"2024-06-03T01:06:35.347435+0000","last_undegraded":"2024-06-03T01:06:35.347435+0000","last_fullsized":"2024-06-03T01:06:35.347435+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:28:46.911683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":89,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:48.410019+0000","last_change":"2024-06-03T01:04:49.781140+0000","last_active":"2024-06-03T01:06:48.410019+0000","last_peered":"2024-06-03T01:06:48.410019+0000","last_clean":"2024-06-03T01:06:48.410019+0000","last_became_active":"2024-06-03T01:04:49.780712+0000","last_became_peered":"2024-06-03T01:04:49.780712+0000","last_unstale":"2024-06-03T01:06:48.410019+0000","last_undegraded":"2024-06-03T01:06:48.410019+0000","last_fullsized":"2024-06-03T01:06:48.410019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:12:48.170997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840344+0000","last_change":"2024-06-03T01:04:54.396470+0000","last_active":"2024-06-03T01:06:35.840344+0000","last_peered":"2024-06-03T01:06:35.840344+0000","last_clean":"2024-06-03T01:06:35.840344+0000","last_became_active":"2024-06-03T01:04:54.394564+0000","last_became_peered":"2024-06-03T01:04:54.394564+0000","last_unstale":"2024-06-03T01:06:35.840344+0000","last_undegraded":"2024-06-03T01:06:35.840344+0000","last_fullsized":"2024-06-03T01:06:35.840344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:21:06.436992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354613+0000","last_change":"2024-06-03T01:05:08.410302+0000","last_active":"2024-06-03T01:06:35.354613+0000","last_peered":"2024-06-03T01:06:35.354613+0000","last_clean":"2024-06-03T01:06:35.354613+0000","last_became_active":"2024-06-03T01:05:08.409971+0000","last_became_peered":"2024-06-03T01:05:08.409971+0000","last_unstale":"2024-06-03T01:06:35.354613+0000","last_undegraded":"2024-06-03T01:06:35.354613+0000","last_fullsized":"2024-06-03T01:06:35.354613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:50.786491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841198+0000","last_change":"2024-06-03T01:04:49.893771+0000","last_active":"2024-06-03T01:06:35.841198+0000","last_peered":"2024-06-03T01:06:35.841198+0000","last_clean":"2024-06-03T01:06:35.841198+0000","last_became_active":"2024-06-03T01:04:49.892855+0000","last_became_peered":"2024-06-03T01:04:49.892855+0000","last_unstale":"2024-06-03T01:06:35.841198+0000","last_undegraded":"2024-06-03T01:06:35.841198+0000","last_fullsized":"2024-06-03T01:06:35.841198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:47.474318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597326+0000","last_change":"2024-06-03T01:04:54.378309+0000","last_active":"2024-06-03T01:06:35.597326+0000","last_peered":"2024-06-03T01:06:35.597326+0000","last_clean":"2024-06-03T01:06:35.597326+0000","last_became_active":"2024-06-03T01:04:54.377926+0000","last_became_peered":"2024-06-03T01:04:54.377926+0000","last_unstale":"2024-06-03T01:06:35.597326+0000","last_undegraded":"2024-06-03T01:06:35.597326+0000","last_fullsized":"2024-06-03T01:06:35.597326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:14:48.919904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.842264+0000","last_change":"2024-06-03T01:06:35.842264+0000","last_active":"2024-06-03T01:06:35.842264+0000","last_peered":"2024-06-03T01:06:35.842264+0000","last_clean":"2024-06-03T01:06:35.842264+0000","last_became_active":"2024-06-03T01:06:35.841786+0000","last_became_peered":"2024-06-03T01:06:35.841786+0000","last_unstale":"2024-06-03T01:06:35.842264+0000","last_undegraded":"2024-06-03T01:06:35.842264+0000","last_fullsized":"2024-06-03T01:06:35.842264+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:15.788405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837952+0000","last_change":"2024-06-03T01:04:49.770978+0000","last_active":"2024-06-03T01:06:35.837952+0000","last_peered":"2024-06-03T01:06:35.837952+0000","last_clean":"2024-06-03T01:06:35.837952+0000","last_became_active":"2024-06-03T01:04:49.770010+0000","last_became_peered":"2024-06-03T01:04:49.770010+0000","last_unstale":"2024-06-03T01:06:35.837952+0000","last_undegraded":"2024-06-03T01:06:35.837952+0000","last_fullsized":"2024-06-03T01:06:35.837952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:33.707218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837984+0000","last_change":"2024-06-03T01:04:54.380616+0000","last_active":"2024-06-03T01:06:35.837984+0000","last_peered":"2024-06-03T01:06:35.837984+0000","last_clean":"2024-06-03T01:06:35.837984+0000","last_became_active":"2024-06-03T01:04:54.380257+0000","last_became_peered":"2024-06-03T01:04:54.380257+0000","last_unstale":"2024-06-03T01:06:35.837984+0000","last_undegraded":"2024-06-03T01:06:35.837984+0000","last_fullsized":"2024-06-03T01:06:35.837984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:14:54.327146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354624+0000","last_change":"2024-06-03T01:05:08.388139+0000","last_active":"2024-06-03T01:06:35.354624+0000","last_peered":"2024-06-03T01:06:35.354624+0000","last_clean":"2024-06-03T01:06:35.354624+0000","last_became_active":"2024-06-03T01:05:08.387697+0000","last_became_peered":"2024-06-03T01:05:08.387697+0000","last_unstale":"2024-06-03T01:06:35.354624+0000","last_undegraded":"2024-06-03T01:06:35.354624+0000","last_fullsized":"2024-06-03T01:06:35.354624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:36:44.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837371+0000","last_change":"2024-06-03T01:04:49.771928+0000","last_active":"2024-06-03T01:06:35.837371+0000","last_peered":"2024-06-03T01:06:35.837371+0000","last_clean":"2024-06-03T01:06:35.837371+0000","last_became_active":"2024-06-03T01:04:49.771517+0000","last_became_peered":"2024-06-03T01:04:49.771517+0000","last_unstale":"2024-06-03T01:06:35.837371+0000","last_undegraded":"2024-06-03T01:06:35.837371+0000","last_fullsized":"2024-06-03T01:06:35.837371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:48:48.147276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351233+0000","last_change":"2024-06-03T01:05:35.261543+0000","last_active":"2024-06-03T01:06:35.351233+0000","last_peered":"2024-06-03T01:06:35.351233+0000","last_clean":"2024-06-03T01:06:35.351233+0000","last_became_active":"2024-06-03T01:05:35.260977+0000","last_became_peered":"2024-06-03T01:05:35.260977+0000","last_unstale":"2024-06-03T01:06:35.351233+0000","last_undegraded":"2024-06-03T01:06:35.351233+0000","last_fullsized":"2024-06-03T01:06:35.351233+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:09:41.748477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347928+0000","last_change":"2024-06-03T01:05:08.387864+0000","last_active":"2024-06-03T01:06:35.347928+0000","last_peered":"2024-06-03T01:06:35.347928+0000","last_clean":"2024-06-03T01:06:35.347928+0000","last_became_active":"2024-06-03T01:05:08.387381+0000","last_became_peered":"2024-06-03T01:05:08.387381+0000","last_unstale":"2024-06-03T01:06:35.347928+0000","last_undegraded":"2024-06-03T01:06:35.347928+0000","last_fullsized":"2024-06-03T01:06:35.347928+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:31.349917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":84,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:48.385206+0000","last_change":"2024-06-03T01:04:49.770745+0000","last_active":"2024-06-03T01:06:48.385206+0000","last_peered":"2024-06-03T01:06:48.385206+0000","last_clean":"2024-06-03T01:06:48.385206+0000","last_became_active":"2024-06-03T01:04:49.770011+0000","last_became_peered":"2024-06-03T01:04:49.770011+0000","last_unstale":"2024-06-03T01:06:48.385206+0000","last_undegraded":"2024-06-03T01:06:48.385206+0000","last_fullsized":"2024-06-03T01:06:48.385206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:58:30.702656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837171+0000","last_change":"2024-06-03T01:04:54.378551+0000","last_active":"2024-06-03T01:06:35.837171+0000","last_peered":"2024-06-03T01:06:35.837171+0000","last_clean":"2024-06-03T01:06:35.837171+0000","last_became_active":"2024-06-03T01:04:54.378201+0000","last_became_peered":"2024-06-03T01:04:54.378201+0000","last_unstale":"2024-06-03T01:06:35.837171+0000","last_undegraded":"2024-06-03T01:06:35.837171+0000","last_fullsized":"2024-06-03T01:06:35.837171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:39:25.918280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347837+0000","last_change":"2024-06-03T01:05:08.407416+0000","last_active":"2024-06-03T01:06:35.347837+0000","last_peered":"2024-06-03T01:06:35.347837+0000","last_clean":"2024-06-03T01:06:35.347837+0000","last_became_active":"2024-06-03T01:05:08.406968+0000","last_became_peered":"2024-06-03T01:05:08.406968+0000","last_unstale":"2024-06-03T01:06:35.347837+0000","last_undegraded":"2024-06-03T01:06:35.347837+0000","last_fullsized":"2024-06-03T01:06:35.347837+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:35.618908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363816+0000","last_change":"2024-06-03T01:04:49.772292+0000","last_active":"2024-06-03T01:06:35.363816+0000","last_peered":"2024-06-03T01:06:35.363816+0000","last_clean":"2024-06-03T01:06:35.363816+0000","last_became_active":"2024-06-03T01:04:49.771964+0000","last_became_peered":"2024-06-03T01:04:49.771964+0000","last_unstale":"2024-06-03T01:06:35.363816+0000","last_undegraded":"2024-06-03T01:06:35.363816+0000","last_fullsized":"2024-06-03T01:06:35.363816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:45.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366871+0000","last_change":"2024-06-03T01:04:54.393864+0000","last_active":"2024-06-03T01:06:35.366871+0000","last_peered":"2024-06-03T01:06:35.366871+0000","last_clean":"2024-06-03T01:06:35.366871+0000","last_became_active":"2024-06-03T01:04:54.393442+0000","last_became_peered":"2024-06-03T01:04:54.393442+0000","last_unstale":"2024-06-03T01:06:35.366871+0000","last_undegraded":"2024-06-03T01:06:35.366871+0000","last_fullsized":"2024-06-03T01:06:35.366871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:29.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.599230+0000","last_change":"2024-06-03T01:06:35.599230+0000","last_active":"2024-06-03T01:06:35.599230+0000","last_peered":"2024-06-03T01:06:35.599230+0000","last_clean":"2024-06-03T01:06:35.599230+0000","last_became_active":"2024-06-03T01:06:35.598811+0000","last_became_peered":"2024-06-03T01:06:35.598811+0000","last_unstale":"2024-06-03T01:06:35.599230+0000","last_undegraded":"2024-06-03T01:06:35.599230+0000","last_fullsized":"2024-06-03T01:06:35.599230+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:50.943560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350397+0000","last_change":"2024-06-03T01:04:49.781942+0000","last_active":"2024-06-03T01:06:35.350397+0000","last_peered":"2024-06-03T01:06:35.350397+0000","last_clean":"2024-06-03T01:06:35.350397+0000","last_became_active":"2024-06-03T01:04:49.781585+0000","last_became_peered":"2024-06-03T01:04:49.781585+0000","last_unstale":"2024-06-03T01:06:35.350397+0000","last_undegraded":"2024-06-03T01:06:35.350397+0000","last_fullsized":"2024-06-03T01:06:35.350397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:06:07.814196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839515+0000","last_change":"2024-06-03T01:04:54.390073+0000","last_active":"2024-06-03T01:06:35.839515+0000","last_peered":"2024-06-03T01:06:35.839515+0000","last_clean":"2024-06-03T01:06:35.839515+0000","last_became_active":"2024-06-03T01:04:54.389698+0000","last_became_peered":"2024-06-03T01:04:54.389698+0000","last_unstale":"2024-06-03T01:06:35.839515+0000","last_undegraded":"2024-06-03T01:06:35.839515+0000","last_fullsized":"2024-06-03T01:06:35.839515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:17:26.447432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597174+0000","last_change":"2024-06-03T01:05:08.395848+0000","last_active":"2024-06-03T01:06:35.597174+0000","last_peered":"2024-06-03T01:06:35.597174+0000","last_clean":"2024-06-03T01:06:35.597174+0000","last_became_active":"2024-06-03T01:05:08.394960+0000","last_became_peered":"2024-06-03T01:05:08.394960+0000","last_unstale":"2024-06-03T01:06:35.597174+0000","last_undegraded":"2024-06-03T01:06:35.597174+0000","last_fullsized":"2024-06-03T01:06:35.597174+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:15.567354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366651+0000","last_change":"2024-06-03T01:04:49.770855+0000","last_active":"2024-06-03T01:06:35.366651+0000","last_peered":"2024-06-03T01:06:35.366651+0000","last_clean":"2024-06-03T01:06:35.366651+0000","last_became_active":"2024-06-03T01:04:49.770553+0000","last_became_peered":"2024-06-03T01:04:49.770553+0000","last_unstale":"2024-06-03T01:06:35.366651+0000","last_undegraded":"2024-06-03T01:06:35.366651+0000","last_fullsized":"2024-06-03T01:06:35.366651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:57.663765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351032+0000","last_change":"2024-06-03T01:04:54.395962+0000","last_active":"2024-06-03T01:06:35.351032+0000","last_peered":"2024-06-03T01:06:35.351032+0000","last_clean":"2024-06-03T01:06:35.351032+0000","last_became_active":"2024-06-03T01:04:54.395463+0000","last_became_peered":"2024-06-03T01:04:54.395463+0000","last_unstale":"2024-06-03T01:06:35.351032+0000","last_undegraded":"2024-06-03T01:06:35.351032+0000","last_fullsized":"2024-06-03T01:06:35.351032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:48.570980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839807+0000","last_change":"2024-06-03T01:05:08.392196+0000","last_active":"2024-06-03T01:06:35.839807+0000","last_peered":"2024-06-03T01:06:35.839807+0000","last_clean":"2024-06-03T01:06:35.839807+0000","last_became_active":"2024-06-03T01:05:08.388980+0000","last_became_peered":"2024-06-03T01:05:08.388980+0000","last_unstale":"2024-06-03T01:06:35.839807+0000","last_undegraded":"2024-06-03T01:06:35.839807+0000","last_fullsized":"2024-06-03T01:06:35.839807+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:01:01.066423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'4","reported_seq":116,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:51.388628+0000","last_change":"2024-06-03T01:04:49.785677+0000","last_active":"2024-06-03T01:06:51.388628+0000","last_peered":"2024-06-03T01:06:51.388628+0000","last_clean":"2024-06-03T01:06:51.388628+0000","last_became_active":"2024-06-03T01:04:49.785306+0000","last_became_peered":"2024-06-03T01:04:49.785306+0000","last_unstale":"2024-06-03T01:06:51.388628+0000","last_undegraded":"2024-06-03T01:06:51.388628+0000","last_fullsized":"2024-06-03T01:06:51.388628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:04:51.861728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838501+0000","last_change":"2024-06-03T01:04:54.370886+0000","last_active":"2024-06-03T01:06:35.838501+0000","last_peered":"2024-06-03T01:06:35.838501+0000","last_clean":"2024-06-03T01:06:35.838501+0000","last_became_active":"2024-06-03T01:04:54.370555+0000","last_became_peered":"2024-06-03T01:04:54.370555+0000","last_unstale":"2024-06-03T01:06:35.838501+0000","last_undegraded":"2024-06-03T01:06:35.838501+0000","last_fullsized":"2024-06-03T01:06:35.838501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:20:23.802391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351470+0000","last_change":"2024-06-03T01:05:08.405735+0000","last_active":"2024-06-03T01:06:35.351470+0000","last_peered":"2024-06-03T01:06:35.351470+0000","last_clean":"2024-06-03T01:06:35.351470+0000","last_became_active":"2024-06-03T01:05:08.405289+0000","last_became_peered":"2024-06-03T01:05:08.405289+0000","last_unstale":"2024-06-03T01:06:35.351470+0000","last_undegraded":"2024-06-03T01:06:35.351470+0000","last_fullsized":"2024-06-03T01:06:35.351470+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:47:52.099470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382438+0000","last_change":"2024-06-03T01:04:49.771708+0000","last_active":"2024-06-03T01:06:35.382438+0000","last_peered":"2024-06-03T01:06:35.382438+0000","last_clean":"2024-06-03T01:06:35.382438+0000","last_became_active":"2024-06-03T01:04:49.771303+0000","last_became_peered":"2024-06-03T01:04:49.771303+0000","last_unstale":"2024-06-03T01:06:35.382438+0000","last_undegraded":"2024-06-03T01:06:35.382438+0000","last_fullsized":"2024-06-03T01:06:35.382438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:15.307453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597764+0000","last_change":"2024-06-03T01:04:54.381206+0000","last_active":"2024-06-03T01:06:35.597764+0000","last_peered":"2024-06-03T01:06:35.597764+0000","last_clean":"2024-06-03T01:06:35.597764+0000","last_became_active":"2024-06-03T01:04:54.380842+0000","last_became_peered":"2024-06-03T01:04:54.380842+0000","last_unstale":"2024-06-03T01:06:35.597764+0000","last_undegraded":"2024-06-03T01:06:35.597764+0000","last_fullsized":"2024-06-03T01:06:35.597764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:37.487354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354949+0000","last_change":"2024-06-03T01:05:08.402554+0000","last_active":"2024-06-03T01:06:35.354949+0000","last_peered":"2024-06-03T01:06:35.354949+0000","last_clean":"2024-06-03T01:06:35.354949+0000","last_became_active":"2024-06-03T01:05:08.402040+0000","last_became_peered":"2024-06-03T01:05:08.402040+0000","last_unstale":"2024-06-03T01:06:35.354949+0000","last_undegraded":"2024-06-03T01:06:35.354949+0000","last_fullsized":"2024-06-03T01:06:35.354949+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:10.713793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366871+0000","last_change":"2024-06-03T01:04:49.895941+0000","last_active":"2024-06-03T01:06:35.366871+0000","last_peered":"2024-06-03T01:06:35.366871+0000","last_clean":"2024-06-03T01:06:35.366871+0000","last_became_active":"2024-06-03T01:04:49.895579+0000","last_became_peered":"2024-06-03T01:04:49.895579+0000","last_unstale":"2024-06-03T01:06:35.366871+0000","last_undegraded":"2024-06-03T01:06:35.366871+0000","last_fullsized":"2024-06-03T01:06:35.366871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:45:31.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838281+0000","last_change":"2024-06-03T01:04:54.369271+0000","last_active":"2024-06-03T01:06:35.838281+0000","last_peered":"2024-06-03T01:06:35.838281+0000","last_clean":"2024-06-03T01:06:35.838281+0000","last_became_active":"2024-06-03T01:04:54.368896+0000","last_became_peered":"2024-06-03T01:04:54.368896+0000","last_unstale":"2024-06-03T01:06:35.838281+0000","last_undegraded":"2024-06-03T01:06:35.838281+0000","last_fullsized":"2024-06-03T01:06:35.838281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:47:54.079642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350872+0000","last_change":"2024-06-03T01:05:08.406394+0000","last_active":"2024-06-03T01:06:35.350872+0000","last_peered":"2024-06-03T01:06:35.350872+0000","last_clean":"2024-06-03T01:06:35.350872+0000","last_became_active":"2024-06-03T01:05:08.405356+0000","last_became_peered":"2024-06-03T01:05:08.405356+0000","last_unstale":"2024-06-03T01:06:35.350872+0000","last_undegraded":"2024-06-03T01:06:35.350872+0000","last_fullsized":"2024-06-03T01:06:35.350872+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:57:40.648759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'37","reported_seq":157,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:51.388609+0000","last_change":"2024-06-03T01:04:49.771275+0000","last_active":"2024-06-03T01:06:51.388609+0000","last_peered":"2024-06-03T01:06:51.388609+0000","last_clean":"2024-06-03T01:06:51.388609+0000","last_became_active":"2024-06-03T01:04:49.770750+0000","last_became_peered":"2024-06-03T01:04:49.770750+0000","last_unstale":"2024-06-03T01:06:51.388609+0000","last_undegraded":"2024-06-03T01:06:51.388609+0000","last_fullsized":"2024-06-03T01:06:51.388609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:27:01.251134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67444,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":104,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837823+0000","last_change":"2024-06-03T01:04:54.364655+0000","last_active":"2024-06-03T01:06:35.837823+0000","last_peered":"2024-06-03T01:06:35.837823+0000","last_clean":"2024-06-03T01:06:35.837823+0000","last_became_active":"2024-06-03T01:04:54.364309+0000","last_became_peered":"2024-06-03T01:04:54.364309+0000","last_unstale":"2024-06-03T01:06:35.837823+0000","last_undegraded":"2024-06-03T01:06:35.837823+0000","last_fullsized":"2024-06-03T01:06:35.837823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:18.141328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.367221+0000","last_change":"2024-06-03T01:05:08.393014+0000","last_active":"2024-06-03T01:06:35.367221+0000","last_peered":"2024-06-03T01:06:35.367221+0000","last_clean":"2024-06-03T01:06:35.367221+0000","last_became_active":"2024-06-03T01:05:08.392614+0000","last_became_peered":"2024-06-03T01:05:08.392614+0000","last_unstale":"2024-06-03T01:06:35.367221+0000","last_undegraded":"2024-06-03T01:06:35.367221+0000","last_fullsized":"2024-06-03T01:06:35.367221+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:36.625154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347730+0000","last_change":"2024-06-03T01:04:49.778551+0000","last_active":"2024-06-03T01:06:35.347730+0000","last_peered":"2024-06-03T01:06:35.347730+0000","last_clean":"2024-06-03T01:06:35.347730+0000","last_became_active":"2024-06-03T01:04:49.778166+0000","last_became_peered":"2024-06-03T01:04:49.778166+0000","last_unstale":"2024-06-03T01:06:35.347730+0000","last_undegraded":"2024-06-03T01:06:35.347730+0000","last_fullsized":"2024-06-03T01:06:35.347730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:02:40.207601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350040+0000","last_change":"2024-06-03T01:04:54.394722+0000","last_active":"2024-06-03T01:06:35.350040+0000","last_peered":"2024-06-03T01:06:35.350040+0000","last_clean":"2024-06-03T01:06:35.350040+0000","last_became_active":"2024-06-03T01:04:54.393548+0000","last_became_peered":"2024-06-03T01:04:54.393548+0000","last_unstale":"2024-06-03T01:06:35.350040+0000","last_undegraded":"2024-06-03T01:06:35.350040+0000","last_fullsized":"2024-06-03T01:06:35.350040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:49.077880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354504+0000","last_change":"2024-06-03T01:05:08.403334+0000","last_active":"2024-06-03T01:06:35.354504+0000","last_peered":"2024-06-03T01:06:35.354504+0000","last_clean":"2024-06-03T01:06:35.354504+0000","last_became_active":"2024-06-03T01:05:08.402835+0000","last_became_peered":"2024-06-03T01:05:08.402835+0000","last_unstale":"2024-06-03T01:06:35.354504+0000","last_undegraded":"2024-06-03T01:06:35.354504+0000","last_fullsized":"2024-06-03T01:06:35.354504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:50:08.551919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366681+0000","last_change":"2024-06-03T01:04:49.782860+0000","last_active":"2024-06-03T01:06:35.366681+0000","last_peered":"2024-06-03T01:06:35.366681+0000","last_clean":"2024-06-03T01:06:35.366681+0000","last_became_active":"2024-06-03T01:04:49.782610+0000","last_became_peered":"2024-06-03T01:04:49.782610+0000","last_unstale":"2024-06-03T01:06:35.366681+0000","last_undegraded":"2024-06-03T01:06:35.366681+0000","last_fullsized":"2024-06-03T01:06:35.366681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:37.967304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382175+0000","last_change":"2024-06-03T01:05:35.253841+0000","last_active":"2024-06-03T01:06:35.382175+0000","last_peered":"2024-06-03T01:06:35.382175+0000","last_clean":"2024-06-03T01:06:35.382175+0000","last_became_active":"2024-06-03T01:05:35.253412+0000","last_became_peered":"2024-06-03T01:05:35.253412+0000","last_unstale":"2024-06-03T01:06:35.382175+0000","last_undegraded":"2024-06-03T01:06:35.382175+0000","last_fullsized":"2024-06-03T01:06:35.382175+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:12:21.890358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837746+0000","last_change":"2024-06-03T01:05:35.259007+0000","last_active":"2024-06-03T01:06:35.837746+0000","last_peered":"2024-06-03T01:06:35.837746+0000","last_clean":"2024-06-03T01:06:35.837746+0000","last_became_active":"2024-06-03T01:05:35.258358+0000","last_became_peered":"2024-06-03T01:05:35.258358+0000","last_unstale":"2024-06-03T01:06:35.837746+0000","last_undegraded":"2024-06-03T01:06:35.837746+0000","last_fullsized":"2024-06-03T01:06:35.837746+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:02.732772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"98'5","reported_seq":83,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:48.390234+0000","last_change":"2024-06-03T01:04:49.771410+0000","last_active":"2024-06-03T01:06:48.390234+0000","last_peered":"2024-06-03T01:06:48.390234+0000","last_clean":"2024-06-03T01:06:48.390234+0000","last_became_active":"2024-06-03T01:04:49.771034+0000","last_became_peered":"2024-06-03T01:04:49.771034+0000","last_unstale":"2024-06-03T01:06:48.390234+0000","last_undegraded":"2024-06-03T01:06:48.390234+0000","last_fullsized":"2024-06-03T01:06:48.390234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:25:30.888075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382383+0000","last_change":"2024-06-03T01:04:54.363918+0000","last_active":"2024-06-03T01:06:35.382383+0000","last_peered":"2024-06-03T01:06:35.382383+0000","last_clean":"2024-06-03T01:06:35.382383+0000","last_became_active":"2024-06-03T01:04:54.363536+0000","last_became_peered":"2024-06-03T01:04:54.363536+0000","last_unstale":"2024-06-03T01:06:35.382383+0000","last_undegraded":"2024-06-03T01:06:35.382383+0000","last_fullsized":"2024-06-03T01:06:35.382383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:22.103375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.353772+0000","last_change":"2024-06-03T01:05:08.403245+0000","last_active":"2024-06-03T01:06:35.353772+0000","last_peered":"2024-06-03T01:06:35.353772+0000","last_clean":"2024-06-03T01:06:35.353772+0000","last_became_active":"2024-06-03T01:05:08.402755+0000","last_became_peered":"2024-06-03T01:05:08.402755+0000","last_unstale":"2024-06-03T01:06:35.353772+0000","last_undegraded":"2024-06-03T01:06:35.353772+0000","last_fullsized":"2024-06-03T01:06:35.353772+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:33.777084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837051+0000","last_change":"2024-06-03T01:04:49.891326+0000","last_active":"2024-06-03T01:06:35.837051+0000","last_peered":"2024-06-03T01:06:35.837051+0000","last_clean":"2024-06-03T01:06:35.837051+0000","last_became_active":"2024-06-03T01:04:49.891040+0000","last_became_peered":"2024-06-03T01:04:49.891040+0000","last_unstale":"2024-06-03T01:06:35.837051+0000","last_undegraded":"2024-06-03T01:06:35.837051+0000","last_fullsized":"2024-06-03T01:06:35.837051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:39:26.896996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366551+0000","last_change":"2024-06-03T01:04:54.372779+0000","last_active":"2024-06-03T01:06:35.366551+0000","last_peered":"2024-06-03T01:06:35.366551+0000","last_clean":"2024-06-03T01:06:35.366551+0000","last_became_active":"2024-06-03T01:04:54.372427+0000","last_became_peered":"2024-06-03T01:04:54.372427+0000","last_unstale":"2024-06-03T01:06:35.366551+0000","last_undegraded":"2024-06-03T01:06:35.366551+0000","last_fullsized":"2024-06-03T01:06:35.366551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:18.837300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366505+0000","last_change":"2024-06-03T01:05:08.391281+0000","last_active":"2024-06-03T01:06:35.366505+0000","last_peered":"2024-06-03T01:06:35.366505+0000","last_clean":"2024-06-03T01:06:35.366505+0000","last_became_active":"2024-06-03T01:05:08.390871+0000","last_became_peered":"2024-06-03T01:05:08.390871+0000","last_unstale":"2024-06-03T01:06:35.366505+0000","last_undegraded":"2024-06-03T01:06:35.366505+0000","last_fullsized":"2024-06-03T01:06:35.366505+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:19:12.174065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837864+0000","last_change":"2024-06-03T01:04:49.769587+0000","last_active":"2024-06-03T01:06:35.837864+0000","last_peered":"2024-06-03T01:06:35.837864+0000","last_clean":"2024-06-03T01:06:35.837864+0000","last_became_active":"2024-06-03T01:04:49.769267+0000","last_became_peered":"2024-06-03T01:04:49.769267+0000","last_unstale":"2024-06-03T01:06:35.837864+0000","last_undegraded":"2024-06-03T01:06:35.837864+0000","last_fullsized":"2024-06-03T01:06:35.837864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:18:30.747021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354882+0000","last_change":"2024-06-03T01:04:54.368293+0000","last_active":"2024-06-03T01:06:35.354882+0000","last_peered":"2024-06-03T01:06:35.354882+0000","last_clean":"2024-06-03T01:06:35.354882+0000","last_became_active":"2024-06-03T01:04:54.367910+0000","last_became_peered":"2024-06-03T01:04:54.367910+0000","last_unstale":"2024-06-03T01:06:35.354882+0000","last_undegraded":"2024-06-03T01:06:35.354882+0000","last_fullsized":"2024-06-03T01:06:35.354882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:23:46.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837109+0000","last_change":"2024-06-03T01:05:08.619467+0000","last_active":"2024-06-03T01:06:35.837109+0000","last_peered":"2024-06-03T01:06:35.837109+0000","last_clean":"2024-06-03T01:06:35.837109+0000","last_became_active":"2024-06-03T01:05:08.618884+0000","last_became_peered":"2024-06-03T01:05:08.618884+0000","last_unstale":"2024-06-03T01:06:35.837109+0000","last_undegraded":"2024-06-03T01:06:35.837109+0000","last_fullsized":"2024-06-03T01:06:35.837109+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:22.072920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352317+0000","last_change":"2024-06-03T01:04:49.896429+0000","last_active":"2024-06-03T01:06:35.352317+0000","last_peered":"2024-06-03T01:06:35.352317+0000","last_clean":"2024-06-03T01:06:35.352317+0000","last_became_active":"2024-06-03T01:04:49.895957+0000","last_became_peered":"2024-06-03T01:04:49.895957+0000","last_unstale":"2024-06-03T01:06:35.352317+0000","last_undegraded":"2024-06-03T01:06:35.352317+0000","last_fullsized":"2024-06-03T01:06:35.352317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:52.362276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.364116+0000","last_change":"2024-06-03T01:04:54.396972+0000","last_active":"2024-06-03T01:06:35.364116+0000","last_peered":"2024-06-03T01:06:35.364116+0000","last_clean":"2024-06-03T01:06:35.364116+0000","last_became_active":"2024-06-03T01:04:54.396737+0000","last_became_peered":"2024-06-03T01:04:54.396737+0000","last_unstale":"2024-06-03T01:06:35.364116+0000","last_undegraded":"2024-06-03T01:06:35.364116+0000","last_fullsized":"2024-06-03T01:06:35.364116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:38:31.722765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351522+0000","last_change":"2024-06-03T01:05:08.390129+0000","last_active":"2024-06-03T01:06:35.351522+0000","last_peered":"2024-06-03T01:06:35.351522+0000","last_clean":"2024-06-03T01:06:35.351522+0000","last_became_active":"2024-06-03T01:05:08.389553+0000","last_became_peered":"2024-06-03T01:05:08.389553+0000","last_unstale":"2024-06-03T01:06:35.351522+0000","last_undegraded":"2024-06-03T01:06:35.351522+0000","last_fullsized":"2024-06-03T01:06:35.351522+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:53:34.176420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'6","reported_seq":88,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:53.420685+0000","last_change":"2024-06-03T01:04:49.777579+0000","last_active":"2024-06-03T01:06:53.420685+0000","last_peered":"2024-06-03T01:06:53.420685+0000","last_clean":"2024-06-03T01:06:53.420685+0000","last_became_active":"2024-06-03T01:04:49.777207+0000","last_became_peered":"2024-06-03T01:04:49.777207+0000","last_unstale":"2024-06-03T01:06:53.420685+0000","last_undegraded":"2024-06-03T01:06:53.420685+0000","last_fullsized":"2024-06-03T01:06:53.420685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:57.941162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363866+0000","last_change":"2024-06-03T01:04:54.382883+0000","last_active":"2024-06-03T01:06:35.363866+0000","last_peered":"2024-06-03T01:06:35.363866+0000","last_clean":"2024-06-03T01:06:35.363866+0000","last_became_active":"2024-06-03T01:04:54.382445+0000","last_became_peered":"2024-06-03T01:04:54.382445+0000","last_unstale":"2024-06-03T01:06:35.363866+0000","last_undegraded":"2024-06-03T01:06:35.363866+0000","last_fullsized":"2024-06-03T01:06:35.363866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:47:07.549646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354400+0000","last_change":"2024-06-03T01:05:08.619324+0000","last_active":"2024-06-03T01:06:35.354400+0000","last_peered":"2024-06-03T01:06:35.354400+0000","last_clean":"2024-06-03T01:06:35.354400+0000","last_became_active":"2024-06-03T01:05:08.618856+0000","last_became_peered":"2024-06-03T01:05:08.618856+0000","last_unstale":"2024-06-03T01:06:35.354400+0000","last_undegraded":"2024-06-03T01:06:35.354400+0000","last_fullsized":"2024-06-03T01:06:35.354400+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:55:49.520528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366987+0000","last_change":"2024-06-03T01:04:54.371883+0000","last_active":"2024-06-03T01:06:35.366987+0000","last_peered":"2024-06-03T01:06:35.366987+0000","last_clean":"2024-06-03T01:06:35.366987+0000","last_became_active":"2024-06-03T01:04:54.371502+0000","last_became_peered":"2024-06-03T01:04:54.371502+0000","last_unstale":"2024-06-03T01:06:35.366987+0000","last_undegraded":"2024-06-03T01:06:35.366987+0000","last_fullsized":"2024-06-03T01:06:35.366987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:14.632927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":80,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:47.388253+0000","last_change":"2024-06-03T01:04:49.893131+0000","last_active":"2024-06-03T01:06:47.388253+0000","last_peered":"2024-06-03T01:06:47.388253+0000","last_clean":"2024-06-03T01:06:47.388253+0000","last_became_active":"2024-06-03T01:04:49.892122+0000","last_became_peered":"2024-06-03T01:04:49.892122+0000","last_unstale":"2024-06-03T01:06:47.388253+0000","last_undegraded":"2024-06-03T01:06:47.388253+0000","last_fullsized":"2024-06-03T01:06:47.388253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:09:07.852255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382539+0000","last_change":"2024-06-03T01:05:08.399018+0000","last_active":"2024-06-03T01:06:35.382539+0000","last_peered":"2024-06-03T01:06:35.382539+0000","last_clean":"2024-06-03T01:06:35.382539+0000","last_became_active":"2024-06-03T01:05:08.398531+0000","last_became_peered":"2024-06-03T01:05:08.398531+0000","last_unstale":"2024-06-03T01:06:35.382539+0000","last_undegraded":"2024-06-03T01:06:35.382539+0000","last_fullsized":"2024-06-03T01:06:35.382539+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:54.115344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840229+0000","last_change":"2024-06-03T01:04:54.385215+0000","last_active":"2024-06-03T01:06:35.840229+0000","last_peered":"2024-06-03T01:06:35.840229+0000","last_clean":"2024-06-03T01:06:35.840229+0000","last_became_active":"2024-06-03T01:04:54.384764+0000","last_became_peered":"2024-06-03T01:04:54.384764+0000","last_unstale":"2024-06-03T01:06:35.840229+0000","last_undegraded":"2024-06-03T01:06:35.840229+0000","last_fullsized":"2024-06-03T01:06:35.840229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:45.388604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354819+0000","last_change":"2024-06-03T01:04:49.781444+0000","last_active":"2024-06-03T01:06:35.354819+0000","last_peered":"2024-06-03T01:06:35.354819+0000","last_clean":"2024-06-03T01:06:35.354819+0000","last_became_active":"2024-06-03T01:04:49.780922+0000","last_became_peered":"2024-06-03T01:04:49.780922+0000","last_unstale":"2024-06-03T01:06:35.354819+0000","last_undegraded":"2024-06-03T01:06:35.354819+0000","last_fullsized":"2024-06-03T01:06:35.354819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:19.843289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":378,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.381996+0000","last_change":"2024-06-03T01:03:16.011282+0000","last_active":"2024-06-03T01:06:35.381996+0000","last_peered":"2024-06-03T01:06:35.381996+0000","last_clean":"2024-06-03T01:06:35.381996+0000","last_became_active":"2024-06-03T01:03:16.010726+0000","last_became_peered":"2024-06-03T01:03:16.010726+0000","last_unstale":"2024-06-03T01:06:35.381996+0000","last_undegraded":"2024-06-03T01:06:35.381996+0000","last_fullsized":"2024-06-03T01:06:35.381996+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_clean_scrub_stamp":"2024-06-03T00:59:20.780015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:50:32.649940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366998+0000","last_change":"2024-06-03T01:05:08.620410+0000","last_active":"2024-06-03T01:06:35.366998+0000","last_peered":"2024-06-03T01:06:35.366998+0000","last_clean":"2024-06-03T01:06:35.366998+0000","last_became_active":"2024-06-03T01:05:08.619601+0000","last_became_peered":"2024-06-03T01:05:08.619601+0000","last_unstale":"2024-06-03T01:06:35.366998+0000","last_undegraded":"2024-06-03T01:06:35.366998+0000","last_fullsized":"2024-06-03T01:06:35.366998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:47.073077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.349746+0000","last_change":"2024-06-03T01:04:54.379373+0000","last_active":"2024-06-03T01:06:35.349746+0000","last_peered":"2024-06-03T01:06:35.349746+0000","last_clean":"2024-06-03T01:06:35.349746+0000","last_became_active":"2024-06-03T01:04:54.378981+0000","last_became_peered":"2024-06-03T01:04:54.378981+0000","last_unstale":"2024-06-03T01:06:35.349746+0000","last_undegraded":"2024-06-03T01:06:35.349746+0000","last_fullsized":"2024-06-03T01:06:35.349746+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:27:06.651091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382211+0000","last_change":"2024-06-03T01:04:49.773896+0000","last_active":"2024-06-03T01:06:35.382211+0000","last_peered":"2024-06-03T01:06:35.382211+0000","last_clean":"2024-06-03T01:06:35.382211+0000","last_became_active":"2024-06-03T01:04:49.773403+0000","last_became_peered":"2024-06-03T01:04:49.773403+0000","last_unstale":"2024-06-03T01:06:35.382211+0000","last_undegraded":"2024-06-03T01:06:35.382211+0000","last_fullsized":"2024-06-03T01:06:35.382211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:34:13.376589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354462+0000","last_change":"2024-06-03T01:05:08.389575+0000","last_active":"2024-06-03T01:06:35.354462+0000","last_peered":"2024-06-03T01:06:35.354462+0000","last_clean":"2024-06-03T01:06:35.354462+0000","last_became_active":"2024-06-03T01:05:08.389010+0000","last_became_peered":"2024-06-03T01:05:08.389010+0000","last_unstale":"2024-06-03T01:06:35.354462+0000","last_undegraded":"2024-06-03T01:06:35.354462+0000","last_fullsized":"2024-06-03T01:06:35.354462+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:57:26.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354667+0000","last_change":"2024-06-03T01:05:35.263346+0000","last_active":"2024-06-03T01:06:35.354667+0000","last_peered":"2024-06-03T01:06:35.354667+0000","last_clean":"2024-06-03T01:06:35.354667+0000","last_became_active":"2024-06-03T01:05:35.262843+0000","last_became_peered":"2024-06-03T01:05:35.262843+0000","last_unstale":"2024-06-03T01:06:35.354667+0000","last_undegraded":"2024-06-03T01:06:35.354667+0000","last_fullsized":"2024-06-03T01:06:35.354667+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:44.247086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352030+0000","last_change":"2024-06-03T01:04:54.388176+0000","last_active":"2024-06-03T01:06:35.352030+0000","last_peered":"2024-06-03T01:06:35.352030+0000","last_clean":"2024-06-03T01:06:35.352030+0000","last_became_active":"2024-06-03T01:04:54.384544+0000","last_became_peered":"2024-06-03T01:04:54.384544+0000","last_unstale":"2024-06-03T01:06:35.352030+0000","last_undegraded":"2024-06-03T01:06:35.352030+0000","last_fullsized":"2024-06-03T01:06:35.352030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:00:42.931031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839494+0000","last_change":"2024-06-03T01:05:08.619540+0000","last_active":"2024-06-03T01:06:35.839494+0000","last_peered":"2024-06-03T01:06:35.839494+0000","last_clean":"2024-06-03T01:06:35.839494+0000","last_became_active":"2024-06-03T01:05:08.618834+0000","last_became_peered":"2024-06-03T01:05:08.618834+0000","last_unstale":"2024-06-03T01:06:35.839494+0000","last_undegraded":"2024-06-03T01:06:35.839494+0000","last_fullsized":"2024-06-03T01:06:35.839494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:07:05.467161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'16","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:44.705536+0000","last_change":"2024-06-03T01:04:49.786635+0000","last_active":"2024-06-03T01:06:44.705536+0000","last_peered":"2024-06-03T01:06:44.705536+0000","last_clean":"2024-06-03T01:06:44.705536+0000","last_became_active":"2024-06-03T01:04:49.785951+0000","last_became_peered":"2024-06-03T01:04:49.785951+0000","last_unstale":"2024-06-03T01:06:44.705536+0000","last_undegraded":"2024-06-03T01:06:44.705536+0000","last_fullsized":"2024-06-03T01:06:44.705536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:10.784322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841187+0000","last_change":"2024-06-03T01:04:54.387111+0000","last_active":"2024-06-03T01:06:35.841187+0000","last_peered":"2024-06-03T01:06:35.841187+0000","last_clean":"2024-06-03T01:06:35.841187+0000","last_became_active":"2024-06-03T01:04:54.384691+0000","last_became_peered":"2024-06-03T01:04:54.384691+0000","last_unstale":"2024-06-03T01:06:35.841187+0000","last_undegraded":"2024-06-03T01:06:35.841187+0000","last_fullsized":"2024-06-03T01:06:35.841187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:11:15.254855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347412+0000","last_change":"2024-06-03T01:05:08.386335+0000","last_active":"2024-06-03T01:06:35.347412+0000","last_peered":"2024-06-03T01:06:35.347412+0000","last_clean":"2024-06-03T01:06:35.347412+0000","last_became_active":"2024-06-03T01:05:08.385722+0000","last_became_peered":"2024-06-03T01:05:08.385722+0000","last_unstale":"2024-06-03T01:06:35.347412+0000","last_undegraded":"2024-06-03T01:06:35.347412+0000","last_fullsized":"2024-06-03T01:06:35.347412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:00.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355271+0000","last_change":"2024-06-03T01:04:49.883125+0000","last_active":"2024-06-03T01:06:35.355271+0000","last_peered":"2024-06-03T01:06:35.355271+0000","last_clean":"2024-06-03T01:06:35.355271+0000","last_became_active":"2024-06-03T01:04:49.882760+0000","last_became_peered":"2024-06-03T01:04:49.882760+0000","last_unstale":"2024-06-03T01:06:35.355271+0000","last_undegraded":"2024-06-03T01:06:35.355271+0000","last_fullsized":"2024-06-03T01:06:35.355271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:05:30.854654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597517+0000","last_change":"2024-06-03T01:04:54.386265+0000","last_active":"2024-06-03T01:06:35.597517+0000","last_peered":"2024-06-03T01:06:35.597517+0000","last_clean":"2024-06-03T01:06:35.597517+0000","last_became_active":"2024-06-03T01:04:54.385801+0000","last_became_peered":"2024-06-03T01:04:54.385801+0000","last_unstale":"2024-06-03T01:06:35.597517+0000","last_undegraded":"2024-06-03T01:06:35.597517+0000","last_fullsized":"2024-06-03T01:06:35.597517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:40:26.128516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354906+0000","last_change":"2024-06-03T01:05:08.619166+0000","last_active":"2024-06-03T01:06:35.354906+0000","last_peered":"2024-06-03T01:06:35.354906+0000","last_clean":"2024-06-03T01:06:35.354906+0000","last_became_active":"2024-06-03T01:05:08.618601+0000","last_became_peered":"2024-06-03T01:05:08.618601+0000","last_unstale":"2024-06-03T01:06:35.354906+0000","last_undegraded":"2024-06-03T01:06:35.354906+0000","last_fullsized":"2024-06-03T01:06:35.354906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:32.401683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837262+0000","last_change":"2024-06-03T01:05:35.258905+0000","last_active":"2024-06-03T01:06:35.837262+0000","last_peered":"2024-06-03T01:06:35.837262+0000","last_clean":"2024-06-03T01:06:35.837262+0000","last_became_active":"2024-06-03T01:05:35.258423+0000","last_became_peered":"2024-06-03T01:05:35.258423+0000","last_unstale":"2024-06-03T01:06:35.837262+0000","last_undegraded":"2024-06-03T01:06:35.837262+0000","last_fullsized":"2024-06-03T01:06:35.837262+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:31:36.969581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837083+0000","last_change":"2024-06-03T01:04:54.367788+0000","last_active":"2024-06-03T01:06:35.837083+0000","last_peered":"2024-06-03T01:06:35.837083+0000","last_clean":"2024-06-03T01:06:35.837083+0000","last_became_active":"2024-06-03T01:04:54.367353+0000","last_became_peered":"2024-06-03T01:04:54.367353+0000","last_unstale":"2024-06-03T01:06:35.837083+0000","last_undegraded":"2024-06-03T01:06:35.837083+0000","last_fullsized":"2024-06-03T01:06:35.837083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:26.604176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352700+0000","last_change":"2024-06-03T01:05:08.405875+0000","last_active":"2024-06-03T01:06:35.352700+0000","last_peered":"2024-06-03T01:06:35.352700+0000","last_clean":"2024-06-03T01:06:35.352700+0000","last_became_active":"2024-06-03T01:05:08.405410+0000","last_became_peered":"2024-06-03T01:05:08.405410+0000","last_unstale":"2024-06-03T01:06:35.352700+0000","last_undegraded":"2024-06-03T01:06:35.352700+0000","last_fullsized":"2024-06-03T01:06:35.352700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:19:51.467150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840032+0000","last_change":"2024-06-03T01:04:49.786446+0000","last_active":"2024-06-03T01:06:35.840032+0000","last_peered":"2024-06-03T01:06:35.840032+0000","last_clean":"2024-06-03T01:06:35.840032+0000","last_became_active":"2024-06-03T01:04:49.786036+0000","last_became_peered":"2024-06-03T01:04:49.786036+0000","last_unstale":"2024-06-03T01:06:35.840032+0000","last_undegraded":"2024-06-03T01:06:35.840032+0000","last_fullsized":"2024-06-03T01:06:35.840032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:41:12.176583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354426+0000","last_change":"2024-06-03T01:04:54.382742+0000","last_active":"2024-06-03T01:06:35.354426+0000","last_peered":"2024-06-03T01:06:35.354426+0000","last_clean":"2024-06-03T01:06:35.354426+0000","last_became_active":"2024-06-03T01:04:54.382296+0000","last_became_peered":"2024-06-03T01:04:54.382296+0000","last_unstale":"2024-06-03T01:06:35.354426+0000","last_undegraded":"2024-06-03T01:06:35.354426+0000","last_fullsized":"2024-06-03T01:06:35.354426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:32:03.385387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"100'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354918+0000","last_change":"2024-06-03T01:05:08.403233+0000","last_active":"2024-06-03T01:06:35.354918+0000","last_peered":"2024-06-03T01:06:35.354918+0000","last_clean":"2024-06-03T01:06:35.354918+0000","last_became_active":"2024-06-03T01:05:08.402693+0000","last_became_peered":"2024-06-03T01:05:08.402693+0000","last_unstale":"2024-06-03T01:06:35.354918+0000","last_undegraded":"2024-06-03T01:06:35.354918+0000","last_fullsized":"2024-06-03T01:06:35.354918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:35:30.722557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347563+0000","last_change":"2024-06-03T01:04:49.779089+0000","last_active":"2024-06-03T01:06:35.347563+0000","last_peered":"2024-06-03T01:06:35.347563+0000","last_clean":"2024-06-03T01:06:35.347563+0000","last_became_active":"2024-06-03T01:04:49.778162+0000","last_became_peered":"2024-06-03T01:04:49.778162+0000","last_unstale":"2024-06-03T01:06:35.347563+0000","last_undegraded":"2024-06-03T01:06:35.347563+0000","last_fullsized":"2024-06-03T01:06:35.347563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:43.151731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840572+0000","last_change":"2024-06-03T01:04:54.393447+0000","last_active":"2024-06-03T01:06:35.840572+0000","last_peered":"2024-06-03T01:06:35.840572+0000","last_clean":"2024-06-03T01:06:35.840572+0000","last_became_active":"2024-06-03T01:04:54.390154+0000","last_became_peered":"2024-06-03T01:04:54.390154+0000","last_unstale":"2024-06-03T01:06:35.840572+0000","last_undegraded":"2024-06-03T01:06:35.840572+0000","last_fullsized":"2024-06-03T01:06:35.840572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:00:21.854391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355057+0000","last_change":"2024-06-03T01:05:08.619603+0000","last_active":"2024-06-03T01:06:35.355057+0000","last_peered":"2024-06-03T01:06:35.355057+0000","last_clean":"2024-06-03T01:06:35.355057+0000","last_became_active":"2024-06-03T01:05:08.618554+0000","last_became_peered":"2024-06-03T01:05:08.618554+0000","last_unstale":"2024-06-03T01:06:35.355057+0000","last_undegraded":"2024-06-03T01:06:35.355057+0000","last_fullsized":"2024-06-03T01:06:35.355057+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:57:26.125390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837379+0000","last_change":"2024-06-03T01:04:49.770531+0000","last_active":"2024-06-03T01:06:35.837379+0000","last_peered":"2024-06-03T01:06:35.837379+0000","last_clean":"2024-06-03T01:06:35.837379+0000","last_became_active":"2024-06-03T01:04:49.770217+0000","last_became_peered":"2024-06-03T01:04:49.770217+0000","last_unstale":"2024-06-03T01:06:35.837379+0000","last_undegraded":"2024-06-03T01:06:35.837379+0000","last_fullsized":"2024-06-03T01:06:35.837379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:59:52.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382251+0000","last_change":"2024-06-03T01:04:54.376511+0000","last_active":"2024-06-03T01:06:35.382251+0000","last_peered":"2024-06-03T01:06:35.382251+0000","last_clean":"2024-06-03T01:06:35.382251+0000","last_became_active":"2024-06-03T01:04:54.375896+0000","last_became_peered":"2024-06-03T01:04:54.375896+0000","last_unstale":"2024-06-03T01:06:35.382251+0000","last_undegraded":"2024-06-03T01:06:35.382251+0000","last_fullsized":"2024-06-03T01:06:35.382251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:20.237450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837308+0000","last_change":"2024-06-03T01:05:08.387551+0000","last_active":"2024-06-03T01:06:35.837308+0000","last_peered":"2024-06-03T01:06:35.837308+0000","last_clean":"2024-06-03T01:06:35.837308+0000","last_became_active":"2024-06-03T01:05:08.386719+0000","last_became_peered":"2024-06-03T01:05:08.386719+0000","last_unstale":"2024-06-03T01:06:35.837308+0000","last_undegraded":"2024-06-03T01:06:35.837308+0000","last_fullsized":"2024-06-03T01:06:35.837308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:03:57.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'6","reported_seq":85,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:48.407327+0000","last_change":"2024-06-03T01:04:49.890435+0000","last_active":"2024-06-03T01:06:48.407327+0000","last_peered":"2024-06-03T01:06:48.407327+0000","last_clean":"2024-06-03T01:06:48.407327+0000","last_became_active":"2024-06-03T01:04:49.890074+0000","last_became_peered":"2024-06-03T01:04:49.890074+0000","last_unstale":"2024-06-03T01:06:48.407327+0000","last_undegraded":"2024-06-03T01:06:48.407327+0000","last_fullsized":"2024-06-03T01:06:48.407327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:48:46.743934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382689+0000","last_change":"2024-06-03T01:04:54.366487+0000","last_active":"2024-06-03T01:06:35.382689+0000","last_peered":"2024-06-03T01:06:35.382689+0000","last_clean":"2024-06-03T01:06:35.382689+0000","last_became_active":"2024-06-03T01:04:54.365905+0000","last_became_peered":"2024-06-03T01:04:54.365905+0000","last_unstale":"2024-06-03T01:06:35.382689+0000","last_undegraded":"2024-06-03T01:06:35.382689+0000","last_fullsized":"2024-06-03T01:06:35.382689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:04:09.159028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596790+0000","last_change":"2024-06-03T01:05:08.404608+0000","last_active":"2024-06-03T01:06:35.596790+0000","last_peered":"2024-06-03T01:06:35.596790+0000","last_clean":"2024-06-03T01:06:35.596790+0000","last_became_active":"2024-06-03T01:05:08.404148+0000","last_became_peered":"2024-06-03T01:05:08.404148+0000","last_unstale":"2024-06-03T01:06:35.596790+0000","last_undegraded":"2024-06-03T01:06:35.596790+0000","last_fullsized":"2024-06-03T01:06:35.596790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:29:54.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366882+0000","last_change":"2024-06-03T01:04:49.778805+0000","last_active":"2024-06-03T01:06:35.366882+0000","last_peered":"2024-06-03T01:06:35.366882+0000","last_clean":"2024-06-03T01:06:35.366882+0000","last_became_active":"2024-06-03T01:04:49.778436+0000","last_became_peered":"2024-06-03T01:04:49.778436+0000","last_unstale":"2024-06-03T01:06:35.366882+0000","last_undegraded":"2024-06-03T01:06:35.366882+0000","last_fullsized":"2024-06-03T01:06:35.366882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:21.369521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382454+0000","last_change":"2024-06-03T01:04:54.393604+0000","last_active":"2024-06-03T01:06:35.382454+0000","last_peered":"2024-06-03T01:06:35.382454+0000","last_clean":"2024-06-03T01:06:35.382454+0000","last_became_active":"2024-06-03T01:04:54.393282+0000","last_became_peered":"2024-06-03T01:04:54.393282+0000","last_unstale":"2024-06-03T01:06:35.382454+0000","last_undegraded":"2024-06-03T01:06:35.382454+0000","last_fullsized":"2024-06-03T01:06:35.382454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:17:24.435502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366357+0000","last_change":"2024-06-03T01:05:08.386399+0000","last_active":"2024-06-03T01:06:35.366357+0000","last_peered":"2024-06-03T01:06:35.366357+0000","last_clean":"2024-06-03T01:06:35.366357+0000","last_became_active":"2024-06-03T01:05:08.384547+0000","last_became_peered":"2024-06-03T01:05:08.384547+0000","last_unstale":"2024-06-03T01:06:35.366357+0000","last_undegraded":"2024-06-03T01:06:35.366357+0000","last_fullsized":"2024-06-03T01:06:35.366357+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:04:17.097695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354077+0000","last_change":"2024-06-03T01:04:49.782877+0000","last_active":"2024-06-03T01:06:35.354077+0000","last_peered":"2024-06-03T01:06:35.354077+0000","last_clean":"2024-06-03T01:06:35.354077+0000","last_became_active":"2024-06-03T01:04:49.781399+0000","last_became_peered":"2024-06-03T01:04:49.781399+0000","last_unstale":"2024-06-03T01:06:35.354077+0000","last_undegraded":"2024-06-03T01:06:35.354077+0000","last_fullsized":"2024-06-03T01:06:35.354077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:16.150699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382647+0000","last_change":"2024-06-03T01:04:54.365914+0000","last_active":"2024-06-03T01:06:35.382647+0000","last_peered":"2024-06-03T01:06:35.382647+0000","last_clean":"2024-06-03T01:06:35.382647+0000","last_became_active":"2024-06-03T01:04:54.365507+0000","last_became_peered":"2024-06-03T01:04:54.365507+0000","last_unstale":"2024-06-03T01:06:35.382647+0000","last_undegraded":"2024-06-03T01:06:35.382647+0000","last_fullsized":"2024-06-03T01:06:35.382647+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:24:04.278102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354505+0000","last_change":"2024-06-03T01:04:54.367546+0000","last_active":"2024-06-03T01:06:35.354505+0000","last_peered":"2024-06-03T01:06:35.354505+0000","last_clean":"2024-06-03T01:06:35.354505+0000","last_became_active":"2024-06-03T01:04:54.367116+0000","last_became_peered":"2024-06-03T01:04:54.367116+0000","last_unstale":"2024-06-03T01:06:35.354505+0000","last_undegraded":"2024-06-03T01:06:35.354505+0000","last_fullsized":"2024-06-03T01:06:35.354505+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:52:15.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347699+0000","last_change":"2024-06-03T01:04:54.366779+0000","last_active":"2024-06-03T01:06:35.347699+0000","last_peered":"2024-06-03T01:06:35.347699+0000","last_clean":"2024-06-03T01:06:35.347699+0000","last_became_active":"2024-06-03T01:04:54.366420+0000","last_became_peered":"2024-06-03T01:04:54.366420+0000","last_unstale":"2024-06-03T01:06:35.347699+0000","last_undegraded":"2024-06-03T01:06:35.347699+0000","last_fullsized":"2024-06-03T01:06:35.347699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:31:16.277064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347522+0000","last_change":"2024-06-03T01:04:54.393396+0000","last_active":"2024-06-03T01:06:35.347522+0000","last_peered":"2024-06-03T01:06:35.347522+0000","last_clean":"2024-06-03T01:06:35.347522+0000","last_became_active":"2024-06-03T01:04:54.393071+0000","last_became_peered":"2024-06-03T01:04:54.393071+0000","last_unstale":"2024-06-03T01:06:35.347522+0000","last_undegraded":"2024-06-03T01:06:35.347522+0000","last_fullsized":"2024-06-03T01:06:35.347522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:25.507494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597465+0000","last_change":"2024-06-03T01:05:35.262012+0000","last_active":"2024-06-03T01:06:35.597465+0000","last_peered":"2024-06-03T01:06:35.597465+0000","last_clean":"2024-06-03T01:06:35.597465+0000","last_became_active":"2024-06-03T01:05:35.261644+0000","last_became_peered":"2024-06-03T01:05:35.261644+0000","last_unstale":"2024-06-03T01:06:35.597465+0000","last_undegraded":"2024-06-03T01:06:35.597465+0000","last_fullsized":"2024-06-03T01:06:35.597465+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:33.480127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351246+0000","last_change":"2024-06-03T01:04:54.388884+0000","last_active":"2024-06-03T01:06:35.351246+0000","last_peered":"2024-06-03T01:06:35.351246+0000","last_clean":"2024-06-03T01:06:35.351246+0000","last_became_active":"2024-06-03T01:04:54.387969+0000","last_became_peered":"2024-06-03T01:04:54.387969+0000","last_unstale":"2024-06-03T01:06:35.351246+0000","last_undegraded":"2024-06-03T01:06:35.351246+0000","last_fullsized":"2024-06-03T01:06:35.351246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:13.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354954+0000","last_change":"2024-06-03T01:04:54.364412+0000","last_active":"2024-06-03T01:06:35.354954+0000","last_peered":"2024-06-03T01:06:35.354954+0000","last_clean":"2024-06-03T01:06:35.354954+0000","last_became_active":"2024-06-03T01:04:54.364053+0000","last_became_peered":"2024-06-03T01:04:54.364053+0000","last_unstale":"2024-06-03T01:06:35.354954+0000","last_undegraded":"2024-06-03T01:06:35.354954+0000","last_fullsized":"2024-06-03T01:06:35.354954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:26.760514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.348068+0000","last_change":"2024-06-03T01:04:54.370604+0000","last_active":"2024-06-03T01:06:35.348068+0000","last_peered":"2024-06-03T01:06:35.348068+0000","last_clean":"2024-06-03T01:06:35.348068+0000","last_became_active":"2024-06-03T01:04:54.370288+0000","last_became_peered":"2024-06-03T01:04:54.370288+0000","last_unstale":"2024-06-03T01:06:35.348068+0000","last_undegraded":"2024-06-03T01:06:35.348068+0000","last_fullsized":"2024-06-03T01:06:35.348068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:34:49.350481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841350+0000","last_change":"2024-06-03T01:05:35.260161+0000","last_active":"2024-06-03T01:06:35.841350+0000","last_peered":"2024-06-03T01:06:35.841350+0000","last_clean":"2024-06-03T01:06:35.841350+0000","last_became_active":"2024-06-03T01:05:35.259725+0000","last_became_peered":"2024-06-03T01:05:35.259725+0000","last_unstale":"2024-06-03T01:06:35.841350+0000","last_undegraded":"2024-06-03T01:06:35.841350+0000","last_fullsized":"2024-06-03T01:06:35.841350+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:36.698726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366656+0000","last_change":"2024-06-03T01:04:54.391161+0000","last_active":"2024-06-03T01:06:35.366656+0000","last_peered":"2024-06-03T01:06:35.366656+0000","last_clean":"2024-06-03T01:06:35.366656+0000","last_became_active":"2024-06-03T01:04:54.390789+0000","last_became_peered":"2024-06-03T01:04:54.390789+0000","last_unstale":"2024-06-03T01:06:35.366656+0000","last_undegraded":"2024-06-03T01:06:35.366656+0000","last_fullsized":"2024-06-03T01:06:35.366656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:54:07.520300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350502+0000","last_change":"2024-06-03T01:05:35.257377+0000","last_active":"2024-06-03T01:06:35.350502+0000","last_peered":"2024-06-03T01:06:35.350502+0000","last_clean":"2024-06-03T01:06:35.350502+0000","last_became_active":"2024-06-03T01:05:35.256864+0000","last_became_peered":"2024-06-03T01:05:35.256864+0000","last_unstale":"2024-06-03T01:06:35.350502+0000","last_undegraded":"2024-06-03T01:06:35.350502+0000","last_fullsized":"2024-06-03T01:06:35.350502+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:06:16.396763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837500+0000","last_change":"2024-06-03T01:04:54.385235+0000","last_active":"2024-06-03T01:06:35.837500+0000","last_peered":"2024-06-03T01:06:35.837500+0000","last_clean":"2024-06-03T01:06:35.837500+0000","last_became_active":"2024-06-03T01:04:54.384834+0000","last_became_peered":"2024-06-03T01:04:54.384834+0000","last_unstale":"2024-06-03T01:06:35.837500+0000","last_undegraded":"2024-06-03T01:06:35.837500+0000","last_fullsized":"2024-06-03T01:06:35.837500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:49.586108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350229+0000","last_change":"2024-06-03T01:04:54.389627+0000","last_active":"2024-06-03T01:06:35.350229+0000","last_peered":"2024-06-03T01:06:35.350229+0000","last_clean":"2024-06-03T01:06:35.350229+0000","last_became_active":"2024-06-03T01:04:54.388430+0000","last_became_peered":"2024-06-03T01:04:54.388430+0000","last_unstale":"2024-06-03T01:06:35.350229+0000","last_undegraded":"2024-06-03T01:06:35.350229+0000","last_fullsized":"2024-06-03T01:06:35.350229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:32:44.376952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354658+0000","last_change":"2024-06-03T01:04:54.366394+0000","last_active":"2024-06-03T01:06:35.354658+0000","last_peered":"2024-06-03T01:06:35.354658+0000","last_clean":"2024-06-03T01:06:35.354658+0000","last_became_active":"2024-06-03T01:04:54.366067+0000","last_became_peered":"2024-06-03T01:04:54.366067+0000","last_unstale":"2024-06-03T01:06:35.354658+0000","last_undegraded":"2024-06-03T01:06:35.354658+0000","last_fullsized":"2024-06-03T01:06:35.354658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:18:03.772654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75036,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":372,"num_read_kb":268,"num_write":253,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":225108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":575353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32992,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712160,"statfs":{"total":95995035648,"available":95961251840,"internally_reserved":0,"allocated":1859584,"data_stored":1229111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":578877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":575047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1777664,"data_stored":1172045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1159168,"data_stored":570675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1232896,"data_stored":639149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1183744,"data_stored":572131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1253376,"data_stored":639759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":569993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":574359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32920,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712232,"statfs":{"total":95995035648,"available":95961325568,"internally_reserved":0,"allocated":1785856,"data_stored":1167237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T01:06:56.015 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T01:06:56.460 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:06:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:56 smithi094 ceph-mon[32119]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:06:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:06:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:06:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:56 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:56 smithi179 ceph-mon[37041]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:06:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:06:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:06:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:56 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:56.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:56 smithi107 ceph-mon[36622]: from='client.24740 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:06:56.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:06:56.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:06:56.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:56 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:06:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:57 smithi094 ceph-mon[32119]: pgmap v401: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-03T01:06:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:57 smithi179 ceph-mon[37041]: pgmap v401: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-03T01:06:57.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:57 smithi107 ceph-mon[36622]: pgmap v401: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-06-03T01:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.711+0000 7f8612000700 1 -- 172.21.15.94:0/2418047184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 msgr2=0x7f860c113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.711+0000 7f8612000700 1 --2- 172.21.15.94:0/2418047184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c113120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009990 tx=0x7f85f4009ca0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.711+0000 7f8612000700 1 -- 172.21.15.94:0/2418047184 shutdown_connections 2024-06-03T01:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.711+0000 7f8612000700 1 --2- 172.21.15.94:0/2418047184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f860c0746c0 0x7f860c074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.712+0000 7f8612000700 1 --2- 172.21.15.94:0/2418047184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860c113660 0x7f860c115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.712+0000 7f8612000700 1 --2- 172.21.15.94:0/2418047184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c113120 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.712+0000 7f8612000700 1 -- 172.21.15.94:0/2418047184 >> 172.21.15.94:0/2418047184 conn(0x7f860c0795f0 msgr2=0x7f860c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.712+0000 7f8612000700 1 -- 172.21.15.94:0/2418047184 shutdown_connections 2024-06-03T01:06:58.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.712+0000 7f8612000700 1 -- 172.21.15.94:0/2418047184 wait complete. 2024-06-03T01:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 Processor -- start 2024-06-03T01:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 -- start start 2024-06-03T01:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860c0746c0 0x7f860c19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f860c113660 0x7f860c1a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:58.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f860c118da0 con 0x7f860c074fe0 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f860c118c20 con 0x7f860c113660 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8612000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f860c118f20 con 0x7f860c0746c0 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f8602a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.713+0000 7f860b400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860c0746c0 0x7f860c19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57730/0 (socket says 172.21.15.94:57730) 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 -- 172.21.15.94:0/1770133966 learned_addr learned my addr 172.21.15.94:0/1770133966 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f860be00700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f860c113660 0x7f860c1a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 -- 172.21.15.94:0/1770133966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860c0746c0 msgr2=0x7f860c19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860c0746c0 0x7f860c19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 -- 172.21.15.94:0/1770133966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f860c113660 msgr2=0x7f860c1a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f860c113660 0x7f860c1a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 -- 172.21.15.94:0/1770133966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c1a42a0 con 0x7f860c074fe0 2024-06-03T01:06:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.714+0000 7f8602a00700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c19f7d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85f400bae0 tx=0x7f85f400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.715+0000 7f8608c00700 1 -- 172.21.15.94:0/1770133966 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4026070 con 0x7f860c074fe0 2024-06-03T01:06:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.715+0000 7f8608c00700 1 -- 172.21.15.94:0/1770133966 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f4003d70 con 0x7f860c074fe0 2024-06-03T01:06:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.715+0000 7f8608c00700 1 -- 172.21.15.94:0/1770133966 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4021410 con 0x7f860c074fe0 2024-06-03T01:06:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.715+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860c1a4530 con 0x7f860c074fe0 2024-06-03T01:06:58.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.715+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860c1abe10 con 0x7f860c074fe0 2024-06-03T01:06:58.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.716+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860c069180 con 0x7f860c074fe0 2024-06-03T01:06:58.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.718+0000 7f8608c00700 1 -- 172.21.15.94:0/1770133966 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85f4029070 con 0x7f860c074fe0 2024-06-03T01:06:58.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.719+0000 7f8608c00700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f85f00a2d30 0x7f85f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:06:58.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.719+0000 7f8608c00700 1 -- 172.21.15.94:0/1770133966 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f85f40d0240 con 0x7f860c074fe0 2024-06-03T01:06:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.720+0000 7f860b400700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f85f00a2d30 0x7f85f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:06:58.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.720+0000 7f860b400700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f85f00a2d30 0x7f85f00a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009c60 tx=0x7f85fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:06:58.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.724+0000 7f8608c00700 1 -- 172.21.15.94:0/1770133966 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f85f405d850 con 0x7f860c074fe0 2024-06-03T01:06:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.945+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f860c112b40 con 0x7f85f00a2d30 2024-06-03T01:06:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.975+0000 7f8608c00700 1 -- 172.21.15.94:0/1770133966 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340534 (secure 0 0 0) 0x7f860c112b40 con 0x7f85f00a2d30 2024-06-03T01:06:58.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:06:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f85f00a2d30 msgr2=0x7f85f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f85f00a2d30 0x7f85f00a51f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009c60 tx=0x7f85fc009260 comp rx=0 tx=0).stop 2024-06-03T01:06:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 msgr2=0x7f860c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:06:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c19f7d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85f400bae0 tx=0x7f85f400bb10 comp rx=0 tx=0).stop 2024-06-03T01:06:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 shutdown_connections 2024-06-03T01:06:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f860c113660 0x7f860c1a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f85f00a2d30 0x7f85f00a51f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f860c0746c0 0x7f860c19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 --2- 172.21.15.94:0/1770133966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f860c074fe0 0x7f860c19f7d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:06:58.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.984+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 >> 172.21.15.94:0/1770133966 conn(0x7f860c0795f0 msgr2=0x7f860c112430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:06:58.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.985+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 shutdown_connections 2024-06-03T01:06:58.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:06:58.985+0000 7f8612000700 1 -- 172.21.15.94:0/1770133966 wait complete. 2024-06-03T01:06:58.986 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-06-03T01:06:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:06:59 smithi094 ceph-mon[32119]: pgmap v402: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-03T01:06:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:06:59 smithi179 ceph-mon[37041]: pgmap v402: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-03T01:06:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:06:59 smithi107 ceph-mon[36622]: pgmap v402: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-06-03T01:06:59.950 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-06-03T01:06:58.312198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665638,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":553,"num_read_kb":412,"num_write":436,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":402348,"kb_used_data":15740,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124539476,"statfs":{"total":1151940427776,"available":1151528423424,"internally_reserved":0,"allocated":16117760,"data_stored":8863736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19581,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":85,"num_write":9,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003762"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355301+0000","last_change":"2024-06-03T01:04:54.382830+0000","last_active":"2024-06-03T01:06:35.355301+0000","last_peered":"2024-06-03T01:06:35.355301+0000","last_clean":"2024-06-03T01:06:35.355301+0000","last_became_active":"2024-06-03T01:04:54.382328+0000","last_became_peered":"2024-06-03T01:04:54.382328+0000","last_unstale":"2024-06-03T01:06:35.355301+0000","last_undegraded":"2024-06-03T01:06:35.355301+0000","last_fullsized":"2024-06-03T01:06:35.355301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:59.037188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597232+0000","last_change":"2024-06-03T01:04:54.380179+0000","last_active":"2024-06-03T01:06:35.597232+0000","last_peered":"2024-06-03T01:06:35.597232+0000","last_clean":"2024-06-03T01:06:35.597232+0000","last_became_active":"2024-06-03T01:04:54.370975+0000","last_became_peered":"2024-06-03T01:04:54.370975+0000","last_unstale":"2024-06-03T01:06:35.597232+0000","last_undegraded":"2024-06-03T01:06:35.597232+0000","last_fullsized":"2024-06-03T01:06:35.597232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:41.133053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596810+0000","last_change":"2024-06-03T01:04:54.381423+0000","last_active":"2024-06-03T01:06:35.596810+0000","last_peered":"2024-06-03T01:06:35.596810+0000","last_clean":"2024-06-03T01:06:35.596810+0000","last_became_active":"2024-06-03T01:04:54.375659+0000","last_became_peered":"2024-06-03T01:04:54.375659+0000","last_unstale":"2024-06-03T01:06:35.596810+0000","last_undegraded":"2024-06-03T01:06:35.596810+0000","last_fullsized":"2024-06-03T01:06:35.596810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:50.294425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355182+0000","last_change":"2024-06-03T01:04:54.372494+0000","last_active":"2024-06-03T01:06:35.355182+0000","last_peered":"2024-06-03T01:06:35.355182+0000","last_clean":"2024-06-03T01:06:35.355182+0000","last_became_active":"2024-06-03T01:04:54.371744+0000","last_became_peered":"2024-06-03T01:04:54.371744+0000","last_unstale":"2024-06-03T01:06:35.355182+0000","last_undegraded":"2024-06-03T01:06:35.355182+0000","last_fullsized":"2024-06-03T01:06:35.355182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:01.887751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837420+0000","last_change":"2024-06-03T01:04:54.394675+0000","last_active":"2024-06-03T01:06:35.837420+0000","last_peered":"2024-06-03T01:06:35.837420+0000","last_clean":"2024-06-03T01:06:35.837420+0000","last_became_active":"2024-06-03T01:04:54.394246+0000","last_became_peered":"2024-06-03T01:04:54.394246+0000","last_unstale":"2024-06-03T01:06:35.837420+0000","last_undegraded":"2024-06-03T01:06:35.837420+0000","last_fullsized":"2024-06-03T01:06:35.837420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:48:47.331598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363884+0000","last_change":"2024-06-03T01:04:54.378765+0000","last_active":"2024-06-03T01:06:35.363884+0000","last_peered":"2024-06-03T01:06:35.363884+0000","last_clean":"2024-06-03T01:06:35.363884+0000","last_became_active":"2024-06-03T01:04:54.378306+0000","last_became_peered":"2024-06-03T01:04:54.378306+0000","last_unstale":"2024-06-03T01:06:35.363884+0000","last_undegraded":"2024-06-03T01:06:35.363884+0000","last_fullsized":"2024-06-03T01:06:35.363884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:26:20.321196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382222+0000","last_change":"2024-06-03T01:05:35.252114+0000","last_active":"2024-06-03T01:06:35.382222+0000","last_peered":"2024-06-03T01:06:35.382222+0000","last_clean":"2024-06-03T01:06:35.382222+0000","last_became_active":"2024-06-03T01:05:35.251849+0000","last_became_peered":"2024-06-03T01:05:35.251849+0000","last_unstale":"2024-06-03T01:06:35.382222+0000","last_undegraded":"2024-06-03T01:06:35.382222+0000","last_fullsized":"2024-06-03T01:06:35.382222+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:18.942723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840012+0000","last_change":"2024-06-03T01:04:54.397423+0000","last_active":"2024-06-03T01:06:35.840012+0000","last_peered":"2024-06-03T01:06:35.840012+0000","last_clean":"2024-06-03T01:06:35.840012+0000","last_became_active":"2024-06-03T01:04:54.394329+0000","last_became_peered":"2024-06-03T01:04:54.394329+0000","last_unstale":"2024-06-03T01:06:35.840012+0000","last_undegraded":"2024-06-03T01:06:35.840012+0000","last_fullsized":"2024-06-03T01:06:35.840012+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:04.020364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838582+0000","last_change":"2024-06-03T01:04:54.368589+0000","last_active":"2024-06-03T01:06:35.838582+0000","last_peered":"2024-06-03T01:06:35.838582+0000","last_clean":"2024-06-03T01:06:35.838582+0000","last_became_active":"2024-06-03T01:04:54.368068+0000","last_became_peered":"2024-06-03T01:04:54.368068+0000","last_unstale":"2024-06-03T01:06:35.838582+0000","last_undegraded":"2024-06-03T01:06:35.838582+0000","last_fullsized":"2024-06-03T01:06:35.838582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:59:44.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.364502+0000","last_change":"2024-06-03T01:04:54.367488+0000","last_active":"2024-06-03T01:06:35.364502+0000","last_peered":"2024-06-03T01:06:35.364502+0000","last_clean":"2024-06-03T01:06:35.364502+0000","last_became_active":"2024-06-03T01:04:54.367022+0000","last_became_peered":"2024-06-03T01:04:54.367022+0000","last_unstale":"2024-06-03T01:06:35.364502+0000","last_undegraded":"2024-06-03T01:06:35.364502+0000","last_fullsized":"2024-06-03T01:06:35.364502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:52:27.697948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382593+0000","last_change":"2024-06-03T01:04:54.366790+0000","last_active":"2024-06-03T01:06:35.382593+0000","last_peered":"2024-06-03T01:06:35.382593+0000","last_clean":"2024-06-03T01:06:35.382593+0000","last_became_active":"2024-06-03T01:04:54.366456+0000","last_became_peered":"2024-06-03T01:04:54.366456+0000","last_unstale":"2024-06-03T01:06:35.382593+0000","last_undegraded":"2024-06-03T01:06:35.382593+0000","last_fullsized":"2024-06-03T01:06:35.382593+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:15:42.495284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355088+0000","last_change":"2024-06-03T01:04:54.369483+0000","last_active":"2024-06-03T01:06:35.355088+0000","last_peered":"2024-06-03T01:06:35.355088+0000","last_clean":"2024-06-03T01:06:35.355088+0000","last_became_active":"2024-06-03T01:04:54.369118+0000","last_became_peered":"2024-06-03T01:04:54.369118+0000","last_unstale":"2024-06-03T01:06:35.355088+0000","last_undegraded":"2024-06-03T01:06:35.355088+0000","last_fullsized":"2024-06-03T01:06:35.355088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:18.218426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350165+0000","last_change":"2024-06-03T01:04:54.380648+0000","last_active":"2024-06-03T01:06:35.350165+0000","last_peered":"2024-06-03T01:06:35.350165+0000","last_clean":"2024-06-03T01:06:35.350165+0000","last_became_active":"2024-06-03T01:04:54.380195+0000","last_became_peered":"2024-06-03T01:04:54.380195+0000","last_unstale":"2024-06-03T01:06:35.350165+0000","last_undegraded":"2024-06-03T01:06:35.350165+0000","last_fullsized":"2024-06-03T01:06:35.350165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:04.991858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596367+0000","last_change":"2024-06-03T01:04:54.367250+0000","last_active":"2024-06-03T01:06:35.596367+0000","last_peered":"2024-06-03T01:06:35.596367+0000","last_clean":"2024-06-03T01:06:35.596367+0000","last_became_active":"2024-06-03T01:04:54.364817+0000","last_became_peered":"2024-06-03T01:04:54.364817+0000","last_unstale":"2024-06-03T01:06:35.596367+0000","last_undegraded":"2024-06-03T01:06:35.596367+0000","last_fullsized":"2024-06-03T01:06:35.596367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:55:20.948692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596342+0000","last_change":"2024-06-03T01:04:54.367640+0000","last_active":"2024-06-03T01:06:35.596342+0000","last_peered":"2024-06-03T01:06:35.596342+0000","last_clean":"2024-06-03T01:06:35.596342+0000","last_became_active":"2024-06-03T01:04:54.367080+0000","last_became_peered":"2024-06-03T01:04:54.367080+0000","last_unstale":"2024-06-03T01:06:35.596342+0000","last_undegraded":"2024-06-03T01:06:35.596342+0000","last_fullsized":"2024-06-03T01:06:35.596342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:43:23.469092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837165+0000","last_change":"2024-06-03T01:04:54.376236+0000","last_active":"2024-06-03T01:06:35.837165+0000","last_peered":"2024-06-03T01:06:35.837165+0000","last_clean":"2024-06-03T01:06:35.837165+0000","last_became_active":"2024-06-03T01:04:54.375866+0000","last_became_peered":"2024-06-03T01:04:54.375866+0000","last_unstale":"2024-06-03T01:06:35.837165+0000","last_undegraded":"2024-06-03T01:06:35.837165+0000","last_fullsized":"2024-06-03T01:06:35.837165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:45.926391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355169+0000","last_change":"2024-06-03T01:04:54.388824+0000","last_active":"2024-06-03T01:06:35.355169+0000","last_peered":"2024-06-03T01:06:35.355169+0000","last_clean":"2024-06-03T01:06:35.355169+0000","last_became_active":"2024-06-03T01:04:54.388491+0000","last_became_peered":"2024-06-03T01:04:54.388491+0000","last_unstale":"2024-06-03T01:06:35.355169+0000","last_undegraded":"2024-06-03T01:06:35.355169+0000","last_fullsized":"2024-06-03T01:06:35.355169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:48.593126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.353313+0000","last_change":"2024-06-03T01:04:54.377623+0000","last_active":"2024-06-03T01:06:35.353313+0000","last_peered":"2024-06-03T01:06:35.353313+0000","last_clean":"2024-06-03T01:06:35.353313+0000","last_became_active":"2024-06-03T01:04:54.377037+0000","last_became_peered":"2024-06-03T01:04:54.377037+0000","last_unstale":"2024-06-03T01:06:35.353313+0000","last_undegraded":"2024-06-03T01:06:35.353313+0000","last_fullsized":"2024-06-03T01:06:35.353313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:52:12.729604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.348167+0000","last_change":"2024-06-03T01:04:54.396946+0000","last_active":"2024-06-03T01:06:35.348167+0000","last_peered":"2024-06-03T01:06:35.348167+0000","last_clean":"2024-06-03T01:06:35.348167+0000","last_became_active":"2024-06-03T01:04:54.396695+0000","last_became_peered":"2024-06-03T01:04:54.396695+0000","last_unstale":"2024-06-03T01:06:35.348167+0000","last_undegraded":"2024-06-03T01:06:35.348167+0000","last_fullsized":"2024-06-03T01:06:35.348167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:10:57.842403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347435+0000","last_change":"2024-06-03T01:05:08.399118+0000","last_active":"2024-06-03T01:06:35.347435+0000","last_peered":"2024-06-03T01:06:35.347435+0000","last_clean":"2024-06-03T01:06:35.347435+0000","last_became_active":"2024-06-03T01:05:08.398716+0000","last_became_peered":"2024-06-03T01:05:08.398716+0000","last_unstale":"2024-06-03T01:06:35.347435+0000","last_undegraded":"2024-06-03T01:06:35.347435+0000","last_fullsized":"2024-06-03T01:06:35.347435+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:28:46.911683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":95,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:54.421555+0000","last_change":"2024-06-03T01:04:49.781140+0000","last_active":"2024-06-03T01:06:54.421555+0000","last_peered":"2024-06-03T01:06:54.421555+0000","last_clean":"2024-06-03T01:06:54.421555+0000","last_became_active":"2024-06-03T01:04:49.780712+0000","last_became_peered":"2024-06-03T01:04:49.780712+0000","last_unstale":"2024-06-03T01:06:54.421555+0000","last_undegraded":"2024-06-03T01:06:54.421555+0000","last_fullsized":"2024-06-03T01:06:54.421555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:12:48.170997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840494+0000","last_change":"2024-06-03T01:04:54.396470+0000","last_active":"2024-06-03T01:06:35.840494+0000","last_peered":"2024-06-03T01:06:35.840494+0000","last_clean":"2024-06-03T01:06:35.840494+0000","last_became_active":"2024-06-03T01:04:54.394564+0000","last_became_peered":"2024-06-03T01:04:54.394564+0000","last_unstale":"2024-06-03T01:06:35.840494+0000","last_undegraded":"2024-06-03T01:06:35.840494+0000","last_fullsized":"2024-06-03T01:06:35.840494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:21:06.436992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354613+0000","last_change":"2024-06-03T01:05:08.410302+0000","last_active":"2024-06-03T01:06:35.354613+0000","last_peered":"2024-06-03T01:06:35.354613+0000","last_clean":"2024-06-03T01:06:35.354613+0000","last_became_active":"2024-06-03T01:05:08.409971+0000","last_became_peered":"2024-06-03T01:05:08.409971+0000","last_unstale":"2024-06-03T01:06:35.354613+0000","last_undegraded":"2024-06-03T01:06:35.354613+0000","last_fullsized":"2024-06-03T01:06:35.354613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:50.786491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841198+0000","last_change":"2024-06-03T01:04:49.893771+0000","last_active":"2024-06-03T01:06:35.841198+0000","last_peered":"2024-06-03T01:06:35.841198+0000","last_clean":"2024-06-03T01:06:35.841198+0000","last_became_active":"2024-06-03T01:04:49.892855+0000","last_became_peered":"2024-06-03T01:04:49.892855+0000","last_unstale":"2024-06-03T01:06:35.841198+0000","last_undegraded":"2024-06-03T01:06:35.841198+0000","last_fullsized":"2024-06-03T01:06:35.841198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:47.474318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597326+0000","last_change":"2024-06-03T01:04:54.378309+0000","last_active":"2024-06-03T01:06:35.597326+0000","last_peered":"2024-06-03T01:06:35.597326+0000","last_clean":"2024-06-03T01:06:35.597326+0000","last_became_active":"2024-06-03T01:04:54.377926+0000","last_became_peered":"2024-06-03T01:04:54.377926+0000","last_unstale":"2024-06-03T01:06:35.597326+0000","last_undegraded":"2024-06-03T01:06:35.597326+0000","last_fullsized":"2024-06-03T01:06:35.597326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:14:48.919904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.842264+0000","last_change":"2024-06-03T01:06:35.842264+0000","last_active":"2024-06-03T01:06:35.842264+0000","last_peered":"2024-06-03T01:06:35.842264+0000","last_clean":"2024-06-03T01:06:35.842264+0000","last_became_active":"2024-06-03T01:06:35.841786+0000","last_became_peered":"2024-06-03T01:06:35.841786+0000","last_unstale":"2024-06-03T01:06:35.842264+0000","last_undegraded":"2024-06-03T01:06:35.842264+0000","last_fullsized":"2024-06-03T01:06:35.842264+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:15.788405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837952+0000","last_change":"2024-06-03T01:04:49.770978+0000","last_active":"2024-06-03T01:06:35.837952+0000","last_peered":"2024-06-03T01:06:35.837952+0000","last_clean":"2024-06-03T01:06:35.837952+0000","last_became_active":"2024-06-03T01:04:49.770010+0000","last_became_peered":"2024-06-03T01:04:49.770010+0000","last_unstale":"2024-06-03T01:06:35.837952+0000","last_undegraded":"2024-06-03T01:06:35.837952+0000","last_fullsized":"2024-06-03T01:06:35.837952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:33.707218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837984+0000","last_change":"2024-06-03T01:04:54.380616+0000","last_active":"2024-06-03T01:06:35.837984+0000","last_peered":"2024-06-03T01:06:35.837984+0000","last_clean":"2024-06-03T01:06:35.837984+0000","last_became_active":"2024-06-03T01:04:54.380257+0000","last_became_peered":"2024-06-03T01:04:54.380257+0000","last_unstale":"2024-06-03T01:06:35.837984+0000","last_undegraded":"2024-06-03T01:06:35.837984+0000","last_fullsized":"2024-06-03T01:06:35.837984+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:14:54.327146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354624+0000","last_change":"2024-06-03T01:05:08.388139+0000","last_active":"2024-06-03T01:06:35.354624+0000","last_peered":"2024-06-03T01:06:35.354624+0000","last_clean":"2024-06-03T01:06:35.354624+0000","last_became_active":"2024-06-03T01:05:08.387697+0000","last_became_peered":"2024-06-03T01:05:08.387697+0000","last_unstale":"2024-06-03T01:06:35.354624+0000","last_undegraded":"2024-06-03T01:06:35.354624+0000","last_fullsized":"2024-06-03T01:06:35.354624+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:36:44.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837371+0000","last_change":"2024-06-03T01:04:49.771928+0000","last_active":"2024-06-03T01:06:35.837371+0000","last_peered":"2024-06-03T01:06:35.837371+0000","last_clean":"2024-06-03T01:06:35.837371+0000","last_became_active":"2024-06-03T01:04:49.771517+0000","last_became_peered":"2024-06-03T01:04:49.771517+0000","last_unstale":"2024-06-03T01:06:35.837371+0000","last_undegraded":"2024-06-03T01:06:35.837371+0000","last_fullsized":"2024-06-03T01:06:35.837371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:48:48.147276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351233+0000","last_change":"2024-06-03T01:05:35.261543+0000","last_active":"2024-06-03T01:06:35.351233+0000","last_peered":"2024-06-03T01:06:35.351233+0000","last_clean":"2024-06-03T01:06:35.351233+0000","last_became_active":"2024-06-03T01:05:35.260977+0000","last_became_peered":"2024-06-03T01:05:35.260977+0000","last_unstale":"2024-06-03T01:06:35.351233+0000","last_undegraded":"2024-06-03T01:06:35.351233+0000","last_fullsized":"2024-06-03T01:06:35.351233+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:09:41.748477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347928+0000","last_change":"2024-06-03T01:05:08.387864+0000","last_active":"2024-06-03T01:06:35.347928+0000","last_peered":"2024-06-03T01:06:35.347928+0000","last_clean":"2024-06-03T01:06:35.347928+0000","last_became_active":"2024-06-03T01:05:08.387381+0000","last_became_peered":"2024-06-03T01:05:08.387381+0000","last_unstale":"2024-06-03T01:06:35.347928+0000","last_undegraded":"2024-06-03T01:06:35.347928+0000","last_fullsized":"2024-06-03T01:06:35.347928+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:31.349917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'6","reported_seq":89,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:53.392169+0000","last_change":"2024-06-03T01:04:49.770745+0000","last_active":"2024-06-03T01:06:53.392169+0000","last_peered":"2024-06-03T01:06:53.392169+0000","last_clean":"2024-06-03T01:06:53.392169+0000","last_became_active":"2024-06-03T01:04:49.770011+0000","last_became_peered":"2024-06-03T01:04:49.770011+0000","last_unstale":"2024-06-03T01:06:53.392169+0000","last_undegraded":"2024-06-03T01:06:53.392169+0000","last_fullsized":"2024-06-03T01:06:53.392169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:58:30.702656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837171+0000","last_change":"2024-06-03T01:04:54.378551+0000","last_active":"2024-06-03T01:06:35.837171+0000","last_peered":"2024-06-03T01:06:35.837171+0000","last_clean":"2024-06-03T01:06:35.837171+0000","last_became_active":"2024-06-03T01:04:54.378201+0000","last_became_peered":"2024-06-03T01:04:54.378201+0000","last_unstale":"2024-06-03T01:06:35.837171+0000","last_undegraded":"2024-06-03T01:06:35.837171+0000","last_fullsized":"2024-06-03T01:06:35.837171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:39:25.918280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347837+0000","last_change":"2024-06-03T01:05:08.407416+0000","last_active":"2024-06-03T01:06:35.347837+0000","last_peered":"2024-06-03T01:06:35.347837+0000","last_clean":"2024-06-03T01:06:35.347837+0000","last_became_active":"2024-06-03T01:05:08.406968+0000","last_became_peered":"2024-06-03T01:05:08.406968+0000","last_unstale":"2024-06-03T01:06:35.347837+0000","last_undegraded":"2024-06-03T01:06:35.347837+0000","last_fullsized":"2024-06-03T01:06:35.347837+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:35.618908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363816+0000","last_change":"2024-06-03T01:04:49.772292+0000","last_active":"2024-06-03T01:06:35.363816+0000","last_peered":"2024-06-03T01:06:35.363816+0000","last_clean":"2024-06-03T01:06:35.363816+0000","last_became_active":"2024-06-03T01:04:49.771964+0000","last_became_peered":"2024-06-03T01:04:49.771964+0000","last_unstale":"2024-06-03T01:06:35.363816+0000","last_undegraded":"2024-06-03T01:06:35.363816+0000","last_fullsized":"2024-06-03T01:06:35.363816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:45.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366871+0000","last_change":"2024-06-03T01:04:54.393864+0000","last_active":"2024-06-03T01:06:35.366871+0000","last_peered":"2024-06-03T01:06:35.366871+0000","last_clean":"2024-06-03T01:06:35.366871+0000","last_became_active":"2024-06-03T01:04:54.393442+0000","last_became_peered":"2024-06-03T01:04:54.393442+0000","last_unstale":"2024-06-03T01:06:35.366871+0000","last_undegraded":"2024-06-03T01:06:35.366871+0000","last_fullsized":"2024-06-03T01:06:35.366871+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:29.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.599230+0000","last_change":"2024-06-03T01:06:35.599230+0000","last_active":"2024-06-03T01:06:35.599230+0000","last_peered":"2024-06-03T01:06:35.599230+0000","last_clean":"2024-06-03T01:06:35.599230+0000","last_became_active":"2024-06-03T01:06:35.598811+0000","last_became_peered":"2024-06-03T01:06:35.598811+0000","last_unstale":"2024-06-03T01:06:35.599230+0000","last_undegraded":"2024-06-03T01:06:35.599230+0000","last_fullsized":"2024-06-03T01:06:35.599230+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:50.943560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350397+0000","last_change":"2024-06-03T01:04:49.781942+0000","last_active":"2024-06-03T01:06:35.350397+0000","last_peered":"2024-06-03T01:06:35.350397+0000","last_clean":"2024-06-03T01:06:35.350397+0000","last_became_active":"2024-06-03T01:04:49.781585+0000","last_became_peered":"2024-06-03T01:04:49.781585+0000","last_unstale":"2024-06-03T01:06:35.350397+0000","last_undegraded":"2024-06-03T01:06:35.350397+0000","last_fullsized":"2024-06-03T01:06:35.350397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:06:07.814196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839515+0000","last_change":"2024-06-03T01:04:54.390073+0000","last_active":"2024-06-03T01:06:35.839515+0000","last_peered":"2024-06-03T01:06:35.839515+0000","last_clean":"2024-06-03T01:06:35.839515+0000","last_became_active":"2024-06-03T01:04:54.389698+0000","last_became_peered":"2024-06-03T01:04:54.389698+0000","last_unstale":"2024-06-03T01:06:35.839515+0000","last_undegraded":"2024-06-03T01:06:35.839515+0000","last_fullsized":"2024-06-03T01:06:35.839515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:17:26.447432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597174+0000","last_change":"2024-06-03T01:05:08.395848+0000","last_active":"2024-06-03T01:06:35.597174+0000","last_peered":"2024-06-03T01:06:35.597174+0000","last_clean":"2024-06-03T01:06:35.597174+0000","last_became_active":"2024-06-03T01:05:08.394960+0000","last_became_peered":"2024-06-03T01:05:08.394960+0000","last_unstale":"2024-06-03T01:06:35.597174+0000","last_undegraded":"2024-06-03T01:06:35.597174+0000","last_fullsized":"2024-06-03T01:06:35.597174+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:15.567354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366651+0000","last_change":"2024-06-03T01:04:49.770855+0000","last_active":"2024-06-03T01:06:35.366651+0000","last_peered":"2024-06-03T01:06:35.366651+0000","last_clean":"2024-06-03T01:06:35.366651+0000","last_became_active":"2024-06-03T01:04:49.770553+0000","last_became_peered":"2024-06-03T01:04:49.770553+0000","last_unstale":"2024-06-03T01:06:35.366651+0000","last_undegraded":"2024-06-03T01:06:35.366651+0000","last_fullsized":"2024-06-03T01:06:35.366651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:57.663765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351032+0000","last_change":"2024-06-03T01:04:54.395962+0000","last_active":"2024-06-03T01:06:35.351032+0000","last_peered":"2024-06-03T01:06:35.351032+0000","last_clean":"2024-06-03T01:06:35.351032+0000","last_became_active":"2024-06-03T01:04:54.395463+0000","last_became_peered":"2024-06-03T01:04:54.395463+0000","last_unstale":"2024-06-03T01:06:35.351032+0000","last_undegraded":"2024-06-03T01:06:35.351032+0000","last_fullsized":"2024-06-03T01:06:35.351032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:48.570980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839807+0000","last_change":"2024-06-03T01:05:08.392196+0000","last_active":"2024-06-03T01:06:35.839807+0000","last_peered":"2024-06-03T01:06:35.839807+0000","last_clean":"2024-06-03T01:06:35.839807+0000","last_became_active":"2024-06-03T01:05:08.388980+0000","last_became_peered":"2024-06-03T01:05:08.388980+0000","last_unstale":"2024-06-03T01:06:35.839807+0000","last_undegraded":"2024-06-03T01:06:35.839807+0000","last_fullsized":"2024-06-03T01:06:35.839807+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:01:01.066423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'4","reported_seq":126,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:56.395563+0000","last_change":"2024-06-03T01:04:49.785677+0000","last_active":"2024-06-03T01:06:56.395563+0000","last_peered":"2024-06-03T01:06:56.395563+0000","last_clean":"2024-06-03T01:06:56.395563+0000","last_became_active":"2024-06-03T01:04:49.785306+0000","last_became_peered":"2024-06-03T01:04:49.785306+0000","last_unstale":"2024-06-03T01:06:56.395563+0000","last_undegraded":"2024-06-03T01:06:56.395563+0000","last_fullsized":"2024-06-03T01:06:56.395563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:04:51.861728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838501+0000","last_change":"2024-06-03T01:04:54.370886+0000","last_active":"2024-06-03T01:06:35.838501+0000","last_peered":"2024-06-03T01:06:35.838501+0000","last_clean":"2024-06-03T01:06:35.838501+0000","last_became_active":"2024-06-03T01:04:54.370555+0000","last_became_peered":"2024-06-03T01:04:54.370555+0000","last_unstale":"2024-06-03T01:06:35.838501+0000","last_undegraded":"2024-06-03T01:06:35.838501+0000","last_fullsized":"2024-06-03T01:06:35.838501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:20:23.802391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351470+0000","last_change":"2024-06-03T01:05:08.405735+0000","last_active":"2024-06-03T01:06:35.351470+0000","last_peered":"2024-06-03T01:06:35.351470+0000","last_clean":"2024-06-03T01:06:35.351470+0000","last_became_active":"2024-06-03T01:05:08.405289+0000","last_became_peered":"2024-06-03T01:05:08.405289+0000","last_unstale":"2024-06-03T01:06:35.351470+0000","last_undegraded":"2024-06-03T01:06:35.351470+0000","last_fullsized":"2024-06-03T01:06:35.351470+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:47:52.099470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382438+0000","last_change":"2024-06-03T01:04:49.771708+0000","last_active":"2024-06-03T01:06:35.382438+0000","last_peered":"2024-06-03T01:06:35.382438+0000","last_clean":"2024-06-03T01:06:35.382438+0000","last_became_active":"2024-06-03T01:04:49.771303+0000","last_became_peered":"2024-06-03T01:04:49.771303+0000","last_unstale":"2024-06-03T01:06:35.382438+0000","last_undegraded":"2024-06-03T01:06:35.382438+0000","last_fullsized":"2024-06-03T01:06:35.382438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:15.307453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597764+0000","last_change":"2024-06-03T01:04:54.381206+0000","last_active":"2024-06-03T01:06:35.597764+0000","last_peered":"2024-06-03T01:06:35.597764+0000","last_clean":"2024-06-03T01:06:35.597764+0000","last_became_active":"2024-06-03T01:04:54.380842+0000","last_became_peered":"2024-06-03T01:04:54.380842+0000","last_unstale":"2024-06-03T01:06:35.597764+0000","last_undegraded":"2024-06-03T01:06:35.597764+0000","last_fullsized":"2024-06-03T01:06:35.597764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:37.487354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354949+0000","last_change":"2024-06-03T01:05:08.402554+0000","last_active":"2024-06-03T01:06:35.354949+0000","last_peered":"2024-06-03T01:06:35.354949+0000","last_clean":"2024-06-03T01:06:35.354949+0000","last_became_active":"2024-06-03T01:05:08.402040+0000","last_became_peered":"2024-06-03T01:05:08.402040+0000","last_unstale":"2024-06-03T01:06:35.354949+0000","last_undegraded":"2024-06-03T01:06:35.354949+0000","last_fullsized":"2024-06-03T01:06:35.354949+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:10.713793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366871+0000","last_change":"2024-06-03T01:04:49.895941+0000","last_active":"2024-06-03T01:06:35.366871+0000","last_peered":"2024-06-03T01:06:35.366871+0000","last_clean":"2024-06-03T01:06:35.366871+0000","last_became_active":"2024-06-03T01:04:49.895579+0000","last_became_peered":"2024-06-03T01:04:49.895579+0000","last_unstale":"2024-06-03T01:06:35.366871+0000","last_undegraded":"2024-06-03T01:06:35.366871+0000","last_fullsized":"2024-06-03T01:06:35.366871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:45:31.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838281+0000","last_change":"2024-06-03T01:04:54.369271+0000","last_active":"2024-06-03T01:06:35.838281+0000","last_peered":"2024-06-03T01:06:35.838281+0000","last_clean":"2024-06-03T01:06:35.838281+0000","last_became_active":"2024-06-03T01:04:54.368896+0000","last_became_peered":"2024-06-03T01:04:54.368896+0000","last_unstale":"2024-06-03T01:06:35.838281+0000","last_undegraded":"2024-06-03T01:06:35.838281+0000","last_fullsized":"2024-06-03T01:06:35.838281+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:47:54.079642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350872+0000","last_change":"2024-06-03T01:05:08.406394+0000","last_active":"2024-06-03T01:06:35.350872+0000","last_peered":"2024-06-03T01:06:35.350872+0000","last_clean":"2024-06-03T01:06:35.350872+0000","last_became_active":"2024-06-03T01:05:08.405356+0000","last_became_peered":"2024-06-03T01:05:08.405356+0000","last_unstale":"2024-06-03T01:06:35.350872+0000","last_undegraded":"2024-06-03T01:06:35.350872+0000","last_fullsized":"2024-06-03T01:06:35.350872+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:57:40.648759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'37","reported_seq":167,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:56.395354+0000","last_change":"2024-06-03T01:04:49.771275+0000","last_active":"2024-06-03T01:06:56.395354+0000","last_peered":"2024-06-03T01:06:56.395354+0000","last_clean":"2024-06-03T01:06:56.395354+0000","last_became_active":"2024-06-03T01:04:49.770750+0000","last_became_peered":"2024-06-03T01:04:49.770750+0000","last_unstale":"2024-06-03T01:06:56.395354+0000","last_undegraded":"2024-06-03T01:06:56.395354+0000","last_fullsized":"2024-06-03T01:06:56.395354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:27:01.251134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67444,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":109,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837823+0000","last_change":"2024-06-03T01:04:54.364655+0000","last_active":"2024-06-03T01:06:35.837823+0000","last_peered":"2024-06-03T01:06:35.837823+0000","last_clean":"2024-06-03T01:06:35.837823+0000","last_became_active":"2024-06-03T01:04:54.364309+0000","last_became_peered":"2024-06-03T01:04:54.364309+0000","last_unstale":"2024-06-03T01:06:35.837823+0000","last_undegraded":"2024-06-03T01:06:35.837823+0000","last_fullsized":"2024-06-03T01:06:35.837823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:18.141328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.367221+0000","last_change":"2024-06-03T01:05:08.393014+0000","last_active":"2024-06-03T01:06:35.367221+0000","last_peered":"2024-06-03T01:06:35.367221+0000","last_clean":"2024-06-03T01:06:35.367221+0000","last_became_active":"2024-06-03T01:05:08.392614+0000","last_became_peered":"2024-06-03T01:05:08.392614+0000","last_unstale":"2024-06-03T01:06:35.367221+0000","last_undegraded":"2024-06-03T01:06:35.367221+0000","last_fullsized":"2024-06-03T01:06:35.367221+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:36.625154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347730+0000","last_change":"2024-06-03T01:04:49.778551+0000","last_active":"2024-06-03T01:06:35.347730+0000","last_peered":"2024-06-03T01:06:35.347730+0000","last_clean":"2024-06-03T01:06:35.347730+0000","last_became_active":"2024-06-03T01:04:49.778166+0000","last_became_peered":"2024-06-03T01:04:49.778166+0000","last_unstale":"2024-06-03T01:06:35.347730+0000","last_undegraded":"2024-06-03T01:06:35.347730+0000","last_fullsized":"2024-06-03T01:06:35.347730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:02:40.207601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350040+0000","last_change":"2024-06-03T01:04:54.394722+0000","last_active":"2024-06-03T01:06:35.350040+0000","last_peered":"2024-06-03T01:06:35.350040+0000","last_clean":"2024-06-03T01:06:35.350040+0000","last_became_active":"2024-06-03T01:04:54.393548+0000","last_became_peered":"2024-06-03T01:04:54.393548+0000","last_unstale":"2024-06-03T01:06:35.350040+0000","last_undegraded":"2024-06-03T01:06:35.350040+0000","last_fullsized":"2024-06-03T01:06:35.350040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:49.077880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354504+0000","last_change":"2024-06-03T01:05:08.403334+0000","last_active":"2024-06-03T01:06:35.354504+0000","last_peered":"2024-06-03T01:06:35.354504+0000","last_clean":"2024-06-03T01:06:35.354504+0000","last_became_active":"2024-06-03T01:05:08.402835+0000","last_became_peered":"2024-06-03T01:05:08.402835+0000","last_unstale":"2024-06-03T01:06:35.354504+0000","last_undegraded":"2024-06-03T01:06:35.354504+0000","last_fullsized":"2024-06-03T01:06:35.354504+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:50:08.551919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366681+0000","last_change":"2024-06-03T01:04:49.782860+0000","last_active":"2024-06-03T01:06:35.366681+0000","last_peered":"2024-06-03T01:06:35.366681+0000","last_clean":"2024-06-03T01:06:35.366681+0000","last_became_active":"2024-06-03T01:04:49.782610+0000","last_became_peered":"2024-06-03T01:04:49.782610+0000","last_unstale":"2024-06-03T01:06:35.366681+0000","last_undegraded":"2024-06-03T01:06:35.366681+0000","last_fullsized":"2024-06-03T01:06:35.366681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:37.967304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382175+0000","last_change":"2024-06-03T01:05:35.253841+0000","last_active":"2024-06-03T01:06:35.382175+0000","last_peered":"2024-06-03T01:06:35.382175+0000","last_clean":"2024-06-03T01:06:35.382175+0000","last_became_active":"2024-06-03T01:05:35.253412+0000","last_became_peered":"2024-06-03T01:05:35.253412+0000","last_unstale":"2024-06-03T01:06:35.382175+0000","last_undegraded":"2024-06-03T01:06:35.382175+0000","last_fullsized":"2024-06-03T01:06:35.382175+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:12:21.890358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837746+0000","last_change":"2024-06-03T01:05:35.259007+0000","last_active":"2024-06-03T01:06:35.837746+0000","last_peered":"2024-06-03T01:06:35.837746+0000","last_clean":"2024-06-03T01:06:35.837746+0000","last_became_active":"2024-06-03T01:05:35.258358+0000","last_became_peered":"2024-06-03T01:05:35.258358+0000","last_unstale":"2024-06-03T01:06:35.837746+0000","last_undegraded":"2024-06-03T01:06:35.837746+0000","last_fullsized":"2024-06-03T01:06:35.837746+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:02.732772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"98'5","reported_seq":88,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:53.399107+0000","last_change":"2024-06-03T01:04:49.771410+0000","last_active":"2024-06-03T01:06:53.399107+0000","last_peered":"2024-06-03T01:06:53.399107+0000","last_clean":"2024-06-03T01:06:53.399107+0000","last_became_active":"2024-06-03T01:04:49.771034+0000","last_became_peered":"2024-06-03T01:04:49.771034+0000","last_unstale":"2024-06-03T01:06:53.399107+0000","last_undegraded":"2024-06-03T01:06:53.399107+0000","last_fullsized":"2024-06-03T01:06:53.399107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:25:30.888075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382383+0000","last_change":"2024-06-03T01:04:54.363918+0000","last_active":"2024-06-03T01:06:35.382383+0000","last_peered":"2024-06-03T01:06:35.382383+0000","last_clean":"2024-06-03T01:06:35.382383+0000","last_became_active":"2024-06-03T01:04:54.363536+0000","last_became_peered":"2024-06-03T01:04:54.363536+0000","last_unstale":"2024-06-03T01:06:35.382383+0000","last_undegraded":"2024-06-03T01:06:35.382383+0000","last_fullsized":"2024-06-03T01:06:35.382383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:22.103375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.353772+0000","last_change":"2024-06-03T01:05:08.403245+0000","last_active":"2024-06-03T01:06:35.353772+0000","last_peered":"2024-06-03T01:06:35.353772+0000","last_clean":"2024-06-03T01:06:35.353772+0000","last_became_active":"2024-06-03T01:05:08.402755+0000","last_became_peered":"2024-06-03T01:05:08.402755+0000","last_unstale":"2024-06-03T01:06:35.353772+0000","last_undegraded":"2024-06-03T01:06:35.353772+0000","last_fullsized":"2024-06-03T01:06:35.353772+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:33.777084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837051+0000","last_change":"2024-06-03T01:04:49.891326+0000","last_active":"2024-06-03T01:06:35.837051+0000","last_peered":"2024-06-03T01:06:35.837051+0000","last_clean":"2024-06-03T01:06:35.837051+0000","last_became_active":"2024-06-03T01:04:49.891040+0000","last_became_peered":"2024-06-03T01:04:49.891040+0000","last_unstale":"2024-06-03T01:06:35.837051+0000","last_undegraded":"2024-06-03T01:06:35.837051+0000","last_fullsized":"2024-06-03T01:06:35.837051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:39:26.896996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366551+0000","last_change":"2024-06-03T01:04:54.372779+0000","last_active":"2024-06-03T01:06:35.366551+0000","last_peered":"2024-06-03T01:06:35.366551+0000","last_clean":"2024-06-03T01:06:35.366551+0000","last_became_active":"2024-06-03T01:04:54.372427+0000","last_became_peered":"2024-06-03T01:04:54.372427+0000","last_unstale":"2024-06-03T01:06:35.366551+0000","last_undegraded":"2024-06-03T01:06:35.366551+0000","last_fullsized":"2024-06-03T01:06:35.366551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:18.837300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366505+0000","last_change":"2024-06-03T01:05:08.391281+0000","last_active":"2024-06-03T01:06:35.366505+0000","last_peered":"2024-06-03T01:06:35.366505+0000","last_clean":"2024-06-03T01:06:35.366505+0000","last_became_active":"2024-06-03T01:05:08.390871+0000","last_became_peered":"2024-06-03T01:05:08.390871+0000","last_unstale":"2024-06-03T01:06:35.366505+0000","last_undegraded":"2024-06-03T01:06:35.366505+0000","last_fullsized":"2024-06-03T01:06:35.366505+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:19:12.174065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837864+0000","last_change":"2024-06-03T01:04:49.769587+0000","last_active":"2024-06-03T01:06:35.837864+0000","last_peered":"2024-06-03T01:06:35.837864+0000","last_clean":"2024-06-03T01:06:35.837864+0000","last_became_active":"2024-06-03T01:04:49.769267+0000","last_became_peered":"2024-06-03T01:04:49.769267+0000","last_unstale":"2024-06-03T01:06:35.837864+0000","last_undegraded":"2024-06-03T01:06:35.837864+0000","last_fullsized":"2024-06-03T01:06:35.837864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:18:30.747021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354882+0000","last_change":"2024-06-03T01:04:54.368293+0000","last_active":"2024-06-03T01:06:35.354882+0000","last_peered":"2024-06-03T01:06:35.354882+0000","last_clean":"2024-06-03T01:06:35.354882+0000","last_became_active":"2024-06-03T01:04:54.367910+0000","last_became_peered":"2024-06-03T01:04:54.367910+0000","last_unstale":"2024-06-03T01:06:35.354882+0000","last_undegraded":"2024-06-03T01:06:35.354882+0000","last_fullsized":"2024-06-03T01:06:35.354882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:23:46.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837109+0000","last_change":"2024-06-03T01:05:08.619467+0000","last_active":"2024-06-03T01:06:35.837109+0000","last_peered":"2024-06-03T01:06:35.837109+0000","last_clean":"2024-06-03T01:06:35.837109+0000","last_became_active":"2024-06-03T01:05:08.618884+0000","last_became_peered":"2024-06-03T01:05:08.618884+0000","last_unstale":"2024-06-03T01:06:35.837109+0000","last_undegraded":"2024-06-03T01:06:35.837109+0000","last_fullsized":"2024-06-03T01:06:35.837109+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:22.072920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352552+0000","last_change":"2024-06-03T01:04:49.896429+0000","last_active":"2024-06-03T01:06:35.352552+0000","last_peered":"2024-06-03T01:06:35.352552+0000","last_clean":"2024-06-03T01:06:35.352552+0000","last_became_active":"2024-06-03T01:04:49.895957+0000","last_became_peered":"2024-06-03T01:04:49.895957+0000","last_unstale":"2024-06-03T01:06:35.352552+0000","last_undegraded":"2024-06-03T01:06:35.352552+0000","last_fullsized":"2024-06-03T01:06:35.352552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:52.362276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.364116+0000","last_change":"2024-06-03T01:04:54.396972+0000","last_active":"2024-06-03T01:06:35.364116+0000","last_peered":"2024-06-03T01:06:35.364116+0000","last_clean":"2024-06-03T01:06:35.364116+0000","last_became_active":"2024-06-03T01:04:54.396737+0000","last_became_peered":"2024-06-03T01:04:54.396737+0000","last_unstale":"2024-06-03T01:06:35.364116+0000","last_undegraded":"2024-06-03T01:06:35.364116+0000","last_fullsized":"2024-06-03T01:06:35.364116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:38:31.722765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351522+0000","last_change":"2024-06-03T01:05:08.390129+0000","last_active":"2024-06-03T01:06:35.351522+0000","last_peered":"2024-06-03T01:06:35.351522+0000","last_clean":"2024-06-03T01:06:35.351522+0000","last_became_active":"2024-06-03T01:05:08.389553+0000","last_became_peered":"2024-06-03T01:05:08.389553+0000","last_unstale":"2024-06-03T01:06:35.351522+0000","last_undegraded":"2024-06-03T01:06:35.351522+0000","last_fullsized":"2024-06-03T01:06:35.351522+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:53:34.176420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'6","reported_seq":88,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:53.420685+0000","last_change":"2024-06-03T01:04:49.777579+0000","last_active":"2024-06-03T01:06:53.420685+0000","last_peered":"2024-06-03T01:06:53.420685+0000","last_clean":"2024-06-03T01:06:53.420685+0000","last_became_active":"2024-06-03T01:04:49.777207+0000","last_became_peered":"2024-06-03T01:04:49.777207+0000","last_unstale":"2024-06-03T01:06:53.420685+0000","last_undegraded":"2024-06-03T01:06:53.420685+0000","last_fullsized":"2024-06-03T01:06:53.420685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:57.941162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363866+0000","last_change":"2024-06-03T01:04:54.382883+0000","last_active":"2024-06-03T01:06:35.363866+0000","last_peered":"2024-06-03T01:06:35.363866+0000","last_clean":"2024-06-03T01:06:35.363866+0000","last_became_active":"2024-06-03T01:04:54.382445+0000","last_became_peered":"2024-06-03T01:04:54.382445+0000","last_unstale":"2024-06-03T01:06:35.363866+0000","last_undegraded":"2024-06-03T01:06:35.363866+0000","last_fullsized":"2024-06-03T01:06:35.363866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:47:07.549646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354400+0000","last_change":"2024-06-03T01:05:08.619324+0000","last_active":"2024-06-03T01:06:35.354400+0000","last_peered":"2024-06-03T01:06:35.354400+0000","last_clean":"2024-06-03T01:06:35.354400+0000","last_became_active":"2024-06-03T01:05:08.618856+0000","last_became_peered":"2024-06-03T01:05:08.618856+0000","last_unstale":"2024-06-03T01:06:35.354400+0000","last_undegraded":"2024-06-03T01:06:35.354400+0000","last_fullsized":"2024-06-03T01:06:35.354400+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:55:49.520528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.367051+0000","last_change":"2024-06-03T01:04:54.371883+0000","last_active":"2024-06-03T01:06:35.367051+0000","last_peered":"2024-06-03T01:06:35.367051+0000","last_clean":"2024-06-03T01:06:35.367051+0000","last_became_active":"2024-06-03T01:04:54.371502+0000","last_became_peered":"2024-06-03T01:04:54.371502+0000","last_unstale":"2024-06-03T01:06:35.367051+0000","last_undegraded":"2024-06-03T01:06:35.367051+0000","last_fullsized":"2024-06-03T01:06:35.367051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:14.632927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":86,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:53.398227+0000","last_change":"2024-06-03T01:04:49.893131+0000","last_active":"2024-06-03T01:06:53.398227+0000","last_peered":"2024-06-03T01:06:53.398227+0000","last_clean":"2024-06-03T01:06:53.398227+0000","last_became_active":"2024-06-03T01:04:49.892122+0000","last_became_peered":"2024-06-03T01:04:49.892122+0000","last_unstale":"2024-06-03T01:06:53.398227+0000","last_undegraded":"2024-06-03T01:06:53.398227+0000","last_fullsized":"2024-06-03T01:06:53.398227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:09:07.852255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382539+0000","last_change":"2024-06-03T01:05:08.399018+0000","last_active":"2024-06-03T01:06:35.382539+0000","last_peered":"2024-06-03T01:06:35.382539+0000","last_clean":"2024-06-03T01:06:35.382539+0000","last_became_active":"2024-06-03T01:05:08.398531+0000","last_became_peered":"2024-06-03T01:05:08.398531+0000","last_unstale":"2024-06-03T01:06:35.382539+0000","last_undegraded":"2024-06-03T01:06:35.382539+0000","last_fullsized":"2024-06-03T01:06:35.382539+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:54.115344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840229+0000","last_change":"2024-06-03T01:04:54.385215+0000","last_active":"2024-06-03T01:06:35.840229+0000","last_peered":"2024-06-03T01:06:35.840229+0000","last_clean":"2024-06-03T01:06:35.840229+0000","last_became_active":"2024-06-03T01:04:54.384764+0000","last_became_peered":"2024-06-03T01:04:54.384764+0000","last_unstale":"2024-06-03T01:06:35.840229+0000","last_undegraded":"2024-06-03T01:06:35.840229+0000","last_fullsized":"2024-06-03T01:06:35.840229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:45.388604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354880+0000","last_change":"2024-06-03T01:04:49.781444+0000","last_active":"2024-06-03T01:06:35.354880+0000","last_peered":"2024-06-03T01:06:35.354880+0000","last_clean":"2024-06-03T01:06:35.354880+0000","last_became_active":"2024-06-03T01:04:49.780922+0000","last_became_peered":"2024-06-03T01:04:49.780922+0000","last_unstale":"2024-06-03T01:06:35.354880+0000","last_undegraded":"2024-06-03T01:06:35.354880+0000","last_fullsized":"2024-06-03T01:06:35.354880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:19.843289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":378,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.381996+0000","last_change":"2024-06-03T01:03:16.011282+0000","last_active":"2024-06-03T01:06:35.381996+0000","last_peered":"2024-06-03T01:06:35.381996+0000","last_clean":"2024-06-03T01:06:35.381996+0000","last_became_active":"2024-06-03T01:03:16.010726+0000","last_became_peered":"2024-06-03T01:03:16.010726+0000","last_unstale":"2024-06-03T01:06:35.381996+0000","last_undegraded":"2024-06-03T01:06:35.381996+0000","last_fullsized":"2024-06-03T01:06:35.381996+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_clean_scrub_stamp":"2024-06-03T00:59:20.780015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:50:32.649940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366998+0000","last_change":"2024-06-03T01:05:08.620410+0000","last_active":"2024-06-03T01:06:35.366998+0000","last_peered":"2024-06-03T01:06:35.366998+0000","last_clean":"2024-06-03T01:06:35.366998+0000","last_became_active":"2024-06-03T01:05:08.619601+0000","last_became_peered":"2024-06-03T01:05:08.619601+0000","last_unstale":"2024-06-03T01:06:35.366998+0000","last_undegraded":"2024-06-03T01:06:35.366998+0000","last_fullsized":"2024-06-03T01:06:35.366998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:47.073077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.349746+0000","last_change":"2024-06-03T01:04:54.379373+0000","last_active":"2024-06-03T01:06:35.349746+0000","last_peered":"2024-06-03T01:06:35.349746+0000","last_clean":"2024-06-03T01:06:35.349746+0000","last_became_active":"2024-06-03T01:04:54.378981+0000","last_became_peered":"2024-06-03T01:04:54.378981+0000","last_unstale":"2024-06-03T01:06:35.349746+0000","last_undegraded":"2024-06-03T01:06:35.349746+0000","last_fullsized":"2024-06-03T01:06:35.349746+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:27:06.651091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382211+0000","last_change":"2024-06-03T01:04:49.773896+0000","last_active":"2024-06-03T01:06:35.382211+0000","last_peered":"2024-06-03T01:06:35.382211+0000","last_clean":"2024-06-03T01:06:35.382211+0000","last_became_active":"2024-06-03T01:04:49.773403+0000","last_became_peered":"2024-06-03T01:04:49.773403+0000","last_unstale":"2024-06-03T01:06:35.382211+0000","last_undegraded":"2024-06-03T01:06:35.382211+0000","last_fullsized":"2024-06-03T01:06:35.382211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:34:13.376589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354462+0000","last_change":"2024-06-03T01:05:08.389575+0000","last_active":"2024-06-03T01:06:35.354462+0000","last_peered":"2024-06-03T01:06:35.354462+0000","last_clean":"2024-06-03T01:06:35.354462+0000","last_became_active":"2024-06-03T01:05:08.389010+0000","last_became_peered":"2024-06-03T01:05:08.389010+0000","last_unstale":"2024-06-03T01:06:35.354462+0000","last_undegraded":"2024-06-03T01:06:35.354462+0000","last_fullsized":"2024-06-03T01:06:35.354462+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:57:26.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354667+0000","last_change":"2024-06-03T01:05:35.263346+0000","last_active":"2024-06-03T01:06:35.354667+0000","last_peered":"2024-06-03T01:06:35.354667+0000","last_clean":"2024-06-03T01:06:35.354667+0000","last_became_active":"2024-06-03T01:05:35.262843+0000","last_became_peered":"2024-06-03T01:05:35.262843+0000","last_unstale":"2024-06-03T01:06:35.354667+0000","last_undegraded":"2024-06-03T01:06:35.354667+0000","last_fullsized":"2024-06-03T01:06:35.354667+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:44.247086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352030+0000","last_change":"2024-06-03T01:04:54.388176+0000","last_active":"2024-06-03T01:06:35.352030+0000","last_peered":"2024-06-03T01:06:35.352030+0000","last_clean":"2024-06-03T01:06:35.352030+0000","last_became_active":"2024-06-03T01:04:54.384544+0000","last_became_peered":"2024-06-03T01:04:54.384544+0000","last_unstale":"2024-06-03T01:06:35.352030+0000","last_undegraded":"2024-06-03T01:06:35.352030+0000","last_fullsized":"2024-06-03T01:06:35.352030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:00:42.931031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839494+0000","last_change":"2024-06-03T01:05:08.619540+0000","last_active":"2024-06-03T01:06:35.839494+0000","last_peered":"2024-06-03T01:06:35.839494+0000","last_clean":"2024-06-03T01:06:35.839494+0000","last_became_active":"2024-06-03T01:05:08.618834+0000","last_became_peered":"2024-06-03T01:05:08.618834+0000","last_unstale":"2024-06-03T01:06:35.839494+0000","last_undegraded":"2024-06-03T01:06:35.839494+0000","last_fullsized":"2024-06-03T01:06:35.839494+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:07:05.467161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'16","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:44.705536+0000","last_change":"2024-06-03T01:04:49.786635+0000","last_active":"2024-06-03T01:06:44.705536+0000","last_peered":"2024-06-03T01:06:44.705536+0000","last_clean":"2024-06-03T01:06:44.705536+0000","last_became_active":"2024-06-03T01:04:49.785951+0000","last_became_peered":"2024-06-03T01:04:49.785951+0000","last_unstale":"2024-06-03T01:06:44.705536+0000","last_undegraded":"2024-06-03T01:06:44.705536+0000","last_fullsized":"2024-06-03T01:06:44.705536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:10.784322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841187+0000","last_change":"2024-06-03T01:04:54.387111+0000","last_active":"2024-06-03T01:06:35.841187+0000","last_peered":"2024-06-03T01:06:35.841187+0000","last_clean":"2024-06-03T01:06:35.841187+0000","last_became_active":"2024-06-03T01:04:54.384691+0000","last_became_peered":"2024-06-03T01:04:54.384691+0000","last_unstale":"2024-06-03T01:06:35.841187+0000","last_undegraded":"2024-06-03T01:06:35.841187+0000","last_fullsized":"2024-06-03T01:06:35.841187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:11:15.254855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347412+0000","last_change":"2024-06-03T01:05:08.386335+0000","last_active":"2024-06-03T01:06:35.347412+0000","last_peered":"2024-06-03T01:06:35.347412+0000","last_clean":"2024-06-03T01:06:35.347412+0000","last_became_active":"2024-06-03T01:05:08.385722+0000","last_became_peered":"2024-06-03T01:05:08.385722+0000","last_unstale":"2024-06-03T01:06:35.347412+0000","last_undegraded":"2024-06-03T01:06:35.347412+0000","last_fullsized":"2024-06-03T01:06:35.347412+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:00.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355271+0000","last_change":"2024-06-03T01:04:49.883125+0000","last_active":"2024-06-03T01:06:35.355271+0000","last_peered":"2024-06-03T01:06:35.355271+0000","last_clean":"2024-06-03T01:06:35.355271+0000","last_became_active":"2024-06-03T01:04:49.882760+0000","last_became_peered":"2024-06-03T01:04:49.882760+0000","last_unstale":"2024-06-03T01:06:35.355271+0000","last_undegraded":"2024-06-03T01:06:35.355271+0000","last_fullsized":"2024-06-03T01:06:35.355271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:05:30.854654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597517+0000","last_change":"2024-06-03T01:04:54.386265+0000","last_active":"2024-06-03T01:06:35.597517+0000","last_peered":"2024-06-03T01:06:35.597517+0000","last_clean":"2024-06-03T01:06:35.597517+0000","last_became_active":"2024-06-03T01:04:54.385801+0000","last_became_peered":"2024-06-03T01:04:54.385801+0000","last_unstale":"2024-06-03T01:06:35.597517+0000","last_undegraded":"2024-06-03T01:06:35.597517+0000","last_fullsized":"2024-06-03T01:06:35.597517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:40:26.128516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354906+0000","last_change":"2024-06-03T01:05:08.619166+0000","last_active":"2024-06-03T01:06:35.354906+0000","last_peered":"2024-06-03T01:06:35.354906+0000","last_clean":"2024-06-03T01:06:35.354906+0000","last_became_active":"2024-06-03T01:05:08.618601+0000","last_became_peered":"2024-06-03T01:05:08.618601+0000","last_unstale":"2024-06-03T01:06:35.354906+0000","last_undegraded":"2024-06-03T01:06:35.354906+0000","last_fullsized":"2024-06-03T01:06:35.354906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:32.401683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837262+0000","last_change":"2024-06-03T01:05:35.258905+0000","last_active":"2024-06-03T01:06:35.837262+0000","last_peered":"2024-06-03T01:06:35.837262+0000","last_clean":"2024-06-03T01:06:35.837262+0000","last_became_active":"2024-06-03T01:05:35.258423+0000","last_became_peered":"2024-06-03T01:05:35.258423+0000","last_unstale":"2024-06-03T01:06:35.837262+0000","last_undegraded":"2024-06-03T01:06:35.837262+0000","last_fullsized":"2024-06-03T01:06:35.837262+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:31:36.969581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837083+0000","last_change":"2024-06-03T01:04:54.367788+0000","last_active":"2024-06-03T01:06:35.837083+0000","last_peered":"2024-06-03T01:06:35.837083+0000","last_clean":"2024-06-03T01:06:35.837083+0000","last_became_active":"2024-06-03T01:04:54.367353+0000","last_became_peered":"2024-06-03T01:04:54.367353+0000","last_unstale":"2024-06-03T01:06:35.837083+0000","last_undegraded":"2024-06-03T01:06:35.837083+0000","last_fullsized":"2024-06-03T01:06:35.837083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:26.604176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352700+0000","last_change":"2024-06-03T01:05:08.405875+0000","last_active":"2024-06-03T01:06:35.352700+0000","last_peered":"2024-06-03T01:06:35.352700+0000","last_clean":"2024-06-03T01:06:35.352700+0000","last_became_active":"2024-06-03T01:05:08.405410+0000","last_became_peered":"2024-06-03T01:05:08.405410+0000","last_unstale":"2024-06-03T01:06:35.352700+0000","last_undegraded":"2024-06-03T01:06:35.352700+0000","last_fullsized":"2024-06-03T01:06:35.352700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:19:51.467150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840032+0000","last_change":"2024-06-03T01:04:49.786446+0000","last_active":"2024-06-03T01:06:35.840032+0000","last_peered":"2024-06-03T01:06:35.840032+0000","last_clean":"2024-06-03T01:06:35.840032+0000","last_became_active":"2024-06-03T01:04:49.786036+0000","last_became_peered":"2024-06-03T01:04:49.786036+0000","last_unstale":"2024-06-03T01:06:35.840032+0000","last_undegraded":"2024-06-03T01:06:35.840032+0000","last_fullsized":"2024-06-03T01:06:35.840032+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:41:12.176583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354426+0000","last_change":"2024-06-03T01:04:54.382742+0000","last_active":"2024-06-03T01:06:35.354426+0000","last_peered":"2024-06-03T01:06:35.354426+0000","last_clean":"2024-06-03T01:06:35.354426+0000","last_became_active":"2024-06-03T01:04:54.382296+0000","last_became_peered":"2024-06-03T01:04:54.382296+0000","last_unstale":"2024-06-03T01:06:35.354426+0000","last_undegraded":"2024-06-03T01:06:35.354426+0000","last_fullsized":"2024-06-03T01:06:35.354426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:32:03.385387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"100'1","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354918+0000","last_change":"2024-06-03T01:05:08.403233+0000","last_active":"2024-06-03T01:06:35.354918+0000","last_peered":"2024-06-03T01:06:35.354918+0000","last_clean":"2024-06-03T01:06:35.354918+0000","last_became_active":"2024-06-03T01:05:08.402693+0000","last_became_peered":"2024-06-03T01:05:08.402693+0000","last_unstale":"2024-06-03T01:06:35.354918+0000","last_undegraded":"2024-06-03T01:06:35.354918+0000","last_fullsized":"2024-06-03T01:06:35.354918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:35:30.722557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347563+0000","last_change":"2024-06-03T01:04:49.779089+0000","last_active":"2024-06-03T01:06:35.347563+0000","last_peered":"2024-06-03T01:06:35.347563+0000","last_clean":"2024-06-03T01:06:35.347563+0000","last_became_active":"2024-06-03T01:04:49.778162+0000","last_became_peered":"2024-06-03T01:04:49.778162+0000","last_unstale":"2024-06-03T01:06:35.347563+0000","last_undegraded":"2024-06-03T01:06:35.347563+0000","last_fullsized":"2024-06-03T01:06:35.347563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:43.151731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840572+0000","last_change":"2024-06-03T01:04:54.393447+0000","last_active":"2024-06-03T01:06:35.840572+0000","last_peered":"2024-06-03T01:06:35.840572+0000","last_clean":"2024-06-03T01:06:35.840572+0000","last_became_active":"2024-06-03T01:04:54.390154+0000","last_became_peered":"2024-06-03T01:04:54.390154+0000","last_unstale":"2024-06-03T01:06:35.840572+0000","last_undegraded":"2024-06-03T01:06:35.840572+0000","last_fullsized":"2024-06-03T01:06:35.840572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:00:21.854391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355057+0000","last_change":"2024-06-03T01:05:08.619603+0000","last_active":"2024-06-03T01:06:35.355057+0000","last_peered":"2024-06-03T01:06:35.355057+0000","last_clean":"2024-06-03T01:06:35.355057+0000","last_became_active":"2024-06-03T01:05:08.618554+0000","last_became_peered":"2024-06-03T01:05:08.618554+0000","last_unstale":"2024-06-03T01:06:35.355057+0000","last_undegraded":"2024-06-03T01:06:35.355057+0000","last_fullsized":"2024-06-03T01:06:35.355057+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:57:26.125390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837379+0000","last_change":"2024-06-03T01:04:49.770531+0000","last_active":"2024-06-03T01:06:35.837379+0000","last_peered":"2024-06-03T01:06:35.837379+0000","last_clean":"2024-06-03T01:06:35.837379+0000","last_became_active":"2024-06-03T01:04:49.770217+0000","last_became_peered":"2024-06-03T01:04:49.770217+0000","last_unstale":"2024-06-03T01:06:35.837379+0000","last_undegraded":"2024-06-03T01:06:35.837379+0000","last_fullsized":"2024-06-03T01:06:35.837379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:59:52.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382251+0000","last_change":"2024-06-03T01:04:54.376511+0000","last_active":"2024-06-03T01:06:35.382251+0000","last_peered":"2024-06-03T01:06:35.382251+0000","last_clean":"2024-06-03T01:06:35.382251+0000","last_became_active":"2024-06-03T01:04:54.375896+0000","last_became_peered":"2024-06-03T01:04:54.375896+0000","last_unstale":"2024-06-03T01:06:35.382251+0000","last_undegraded":"2024-06-03T01:06:35.382251+0000","last_fullsized":"2024-06-03T01:06:35.382251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:20.237450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837308+0000","last_change":"2024-06-03T01:05:08.387551+0000","last_active":"2024-06-03T01:06:35.837308+0000","last_peered":"2024-06-03T01:06:35.837308+0000","last_clean":"2024-06-03T01:06:35.837308+0000","last_became_active":"2024-06-03T01:05:08.386719+0000","last_became_peered":"2024-06-03T01:05:08.386719+0000","last_unstale":"2024-06-03T01:06:35.837308+0000","last_undegraded":"2024-06-03T01:06:35.837308+0000","last_fullsized":"2024-06-03T01:06:35.837308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:03:57.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'6","reported_seq":90,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:53.416719+0000","last_change":"2024-06-03T01:04:49.890435+0000","last_active":"2024-06-03T01:06:53.416719+0000","last_peered":"2024-06-03T01:06:53.416719+0000","last_clean":"2024-06-03T01:06:53.416719+0000","last_became_active":"2024-06-03T01:04:49.890074+0000","last_became_peered":"2024-06-03T01:04:49.890074+0000","last_unstale":"2024-06-03T01:06:53.416719+0000","last_undegraded":"2024-06-03T01:06:53.416719+0000","last_fullsized":"2024-06-03T01:06:53.416719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:48:46.743934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382689+0000","last_change":"2024-06-03T01:04:54.366487+0000","last_active":"2024-06-03T01:06:35.382689+0000","last_peered":"2024-06-03T01:06:35.382689+0000","last_clean":"2024-06-03T01:06:35.382689+0000","last_became_active":"2024-06-03T01:04:54.365905+0000","last_became_peered":"2024-06-03T01:04:54.365905+0000","last_unstale":"2024-06-03T01:06:35.382689+0000","last_undegraded":"2024-06-03T01:06:35.382689+0000","last_fullsized":"2024-06-03T01:06:35.382689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:04:09.159028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.596790+0000","last_change":"2024-06-03T01:05:08.404608+0000","last_active":"2024-06-03T01:06:35.596790+0000","last_peered":"2024-06-03T01:06:35.596790+0000","last_clean":"2024-06-03T01:06:35.596790+0000","last_became_active":"2024-06-03T01:05:08.404148+0000","last_became_peered":"2024-06-03T01:05:08.404148+0000","last_unstale":"2024-06-03T01:06:35.596790+0000","last_undegraded":"2024-06-03T01:06:35.596790+0000","last_fullsized":"2024-06-03T01:06:35.596790+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:29:54.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366882+0000","last_change":"2024-06-03T01:04:49.778805+0000","last_active":"2024-06-03T01:06:35.366882+0000","last_peered":"2024-06-03T01:06:35.366882+0000","last_clean":"2024-06-03T01:06:35.366882+0000","last_became_active":"2024-06-03T01:04:49.778436+0000","last_became_peered":"2024-06-03T01:04:49.778436+0000","last_unstale":"2024-06-03T01:06:35.366882+0000","last_undegraded":"2024-06-03T01:06:35.366882+0000","last_fullsized":"2024-06-03T01:06:35.366882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:21.369521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382454+0000","last_change":"2024-06-03T01:04:54.393604+0000","last_active":"2024-06-03T01:06:35.382454+0000","last_peered":"2024-06-03T01:06:35.382454+0000","last_clean":"2024-06-03T01:06:35.382454+0000","last_became_active":"2024-06-03T01:04:54.393282+0000","last_became_peered":"2024-06-03T01:04:54.393282+0000","last_unstale":"2024-06-03T01:06:35.382454+0000","last_undegraded":"2024-06-03T01:06:35.382454+0000","last_fullsized":"2024-06-03T01:06:35.382454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:17:24.435502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366357+0000","last_change":"2024-06-03T01:05:08.386399+0000","last_active":"2024-06-03T01:06:35.366357+0000","last_peered":"2024-06-03T01:06:35.366357+0000","last_clean":"2024-06-03T01:06:35.366357+0000","last_became_active":"2024-06-03T01:05:08.384547+0000","last_became_peered":"2024-06-03T01:05:08.384547+0000","last_unstale":"2024-06-03T01:06:35.366357+0000","last_undegraded":"2024-06-03T01:06:35.366357+0000","last_fullsized":"2024-06-03T01:06:35.366357+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:04:17.097695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354077+0000","last_change":"2024-06-03T01:04:49.782877+0000","last_active":"2024-06-03T01:06:35.354077+0000","last_peered":"2024-06-03T01:06:35.354077+0000","last_clean":"2024-06-03T01:06:35.354077+0000","last_became_active":"2024-06-03T01:04:49.781399+0000","last_became_peered":"2024-06-03T01:04:49.781399+0000","last_unstale":"2024-06-03T01:06:35.354077+0000","last_undegraded":"2024-06-03T01:06:35.354077+0000","last_fullsized":"2024-06-03T01:06:35.354077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:16.150699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382647+0000","last_change":"2024-06-03T01:04:54.365914+0000","last_active":"2024-06-03T01:06:35.382647+0000","last_peered":"2024-06-03T01:06:35.382647+0000","last_clean":"2024-06-03T01:06:35.382647+0000","last_became_active":"2024-06-03T01:04:54.365507+0000","last_became_peered":"2024-06-03T01:04:54.365507+0000","last_unstale":"2024-06-03T01:06:35.382647+0000","last_undegraded":"2024-06-03T01:06:35.382647+0000","last_fullsized":"2024-06-03T01:06:35.382647+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:24:04.278102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354505+0000","last_change":"2024-06-03T01:04:54.367546+0000","last_active":"2024-06-03T01:06:35.354505+0000","last_peered":"2024-06-03T01:06:35.354505+0000","last_clean":"2024-06-03T01:06:35.354505+0000","last_became_active":"2024-06-03T01:04:54.367116+0000","last_became_peered":"2024-06-03T01:04:54.367116+0000","last_unstale":"2024-06-03T01:06:35.354505+0000","last_undegraded":"2024-06-03T01:06:35.354505+0000","last_fullsized":"2024-06-03T01:06:35.354505+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:52:15.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347699+0000","last_change":"2024-06-03T01:04:54.366779+0000","last_active":"2024-06-03T01:06:35.347699+0000","last_peered":"2024-06-03T01:06:35.347699+0000","last_clean":"2024-06-03T01:06:35.347699+0000","last_became_active":"2024-06-03T01:04:54.366420+0000","last_became_peered":"2024-06-03T01:04:54.366420+0000","last_unstale":"2024-06-03T01:06:35.347699+0000","last_undegraded":"2024-06-03T01:06:35.347699+0000","last_fullsized":"2024-06-03T01:06:35.347699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:31:16.277064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347522+0000","last_change":"2024-06-03T01:04:54.393396+0000","last_active":"2024-06-03T01:06:35.347522+0000","last_peered":"2024-06-03T01:06:35.347522+0000","last_clean":"2024-06-03T01:06:35.347522+0000","last_became_active":"2024-06-03T01:04:54.393071+0000","last_became_peered":"2024-06-03T01:04:54.393071+0000","last_unstale":"2024-06-03T01:06:35.347522+0000","last_undegraded":"2024-06-03T01:06:35.347522+0000","last_fullsized":"2024-06-03T01:06:35.347522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:25.507494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597465+0000","last_change":"2024-06-03T01:05:35.262012+0000","last_active":"2024-06-03T01:06:35.597465+0000","last_peered":"2024-06-03T01:06:35.597465+0000","last_clean":"2024-06-03T01:06:35.597465+0000","last_became_active":"2024-06-03T01:05:35.261644+0000","last_became_peered":"2024-06-03T01:05:35.261644+0000","last_unstale":"2024-06-03T01:06:35.597465+0000","last_undegraded":"2024-06-03T01:06:35.597465+0000","last_fullsized":"2024-06-03T01:06:35.597465+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:33.480127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351246+0000","last_change":"2024-06-03T01:04:54.388884+0000","last_active":"2024-06-03T01:06:35.351246+0000","last_peered":"2024-06-03T01:06:35.351246+0000","last_clean":"2024-06-03T01:06:35.351246+0000","last_became_active":"2024-06-03T01:04:54.387969+0000","last_became_peered":"2024-06-03T01:04:54.387969+0000","last_unstale":"2024-06-03T01:06:35.351246+0000","last_undegraded":"2024-06-03T01:06:35.351246+0000","last_fullsized":"2024-06-03T01:06:35.351246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:13.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354954+0000","last_change":"2024-06-03T01:04:54.364412+0000","last_active":"2024-06-03T01:06:35.354954+0000","last_peered":"2024-06-03T01:06:35.354954+0000","last_clean":"2024-06-03T01:06:35.354954+0000","last_became_active":"2024-06-03T01:04:54.364053+0000","last_became_peered":"2024-06-03T01:04:54.364053+0000","last_unstale":"2024-06-03T01:06:35.354954+0000","last_undegraded":"2024-06-03T01:06:35.354954+0000","last_fullsized":"2024-06-03T01:06:35.354954+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:26.760514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.348068+0000","last_change":"2024-06-03T01:04:54.370604+0000","last_active":"2024-06-03T01:06:35.348068+0000","last_peered":"2024-06-03T01:06:35.348068+0000","last_clean":"2024-06-03T01:06:35.348068+0000","last_became_active":"2024-06-03T01:04:54.370288+0000","last_became_peered":"2024-06-03T01:04:54.370288+0000","last_unstale":"2024-06-03T01:06:35.348068+0000","last_undegraded":"2024-06-03T01:06:35.348068+0000","last_fullsized":"2024-06-03T01:06:35.348068+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:34:49.350481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841350+0000","last_change":"2024-06-03T01:05:35.260161+0000","last_active":"2024-06-03T01:06:35.841350+0000","last_peered":"2024-06-03T01:06:35.841350+0000","last_clean":"2024-06-03T01:06:35.841350+0000","last_became_active":"2024-06-03T01:05:35.259725+0000","last_became_peered":"2024-06-03T01:05:35.259725+0000","last_unstale":"2024-06-03T01:06:35.841350+0000","last_undegraded":"2024-06-03T01:06:35.841350+0000","last_fullsized":"2024-06-03T01:06:35.841350+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:36.698726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366656+0000","last_change":"2024-06-03T01:04:54.391161+0000","last_active":"2024-06-03T01:06:35.366656+0000","last_peered":"2024-06-03T01:06:35.366656+0000","last_clean":"2024-06-03T01:06:35.366656+0000","last_became_active":"2024-06-03T01:04:54.390789+0000","last_became_peered":"2024-06-03T01:04:54.390789+0000","last_unstale":"2024-06-03T01:06:35.366656+0000","last_undegraded":"2024-06-03T01:06:35.366656+0000","last_fullsized":"2024-06-03T01:06:35.366656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:54:07.520300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350502+0000","last_change":"2024-06-03T01:05:35.257377+0000","last_active":"2024-06-03T01:06:35.350502+0000","last_peered":"2024-06-03T01:06:35.350502+0000","last_clean":"2024-06-03T01:06:35.350502+0000","last_became_active":"2024-06-03T01:05:35.256864+0000","last_became_peered":"2024-06-03T01:05:35.256864+0000","last_unstale":"2024-06-03T01:06:35.350502+0000","last_undegraded":"2024-06-03T01:06:35.350502+0000","last_fullsized":"2024-06-03T01:06:35.350502+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:06:16.396763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837500+0000","last_change":"2024-06-03T01:04:54.385235+0000","last_active":"2024-06-03T01:06:35.837500+0000","last_peered":"2024-06-03T01:06:35.837500+0000","last_clean":"2024-06-03T01:06:35.837500+0000","last_became_active":"2024-06-03T01:04:54.384834+0000","last_became_peered":"2024-06-03T01:04:54.384834+0000","last_unstale":"2024-06-03T01:06:35.837500+0000","last_undegraded":"2024-06-03T01:06:35.837500+0000","last_fullsized":"2024-06-03T01:06:35.837500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:49.586108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350229+0000","last_change":"2024-06-03T01:04:54.389627+0000","last_active":"2024-06-03T01:06:35.350229+0000","last_peered":"2024-06-03T01:06:35.350229+0000","last_clean":"2024-06-03T01:06:35.350229+0000","last_became_active":"2024-06-03T01:04:54.388430+0000","last_became_peered":"2024-06-03T01:04:54.388430+0000","last_unstale":"2024-06-03T01:06:35.350229+0000","last_undegraded":"2024-06-03T01:06:35.350229+0000","last_fullsized":"2024-06-03T01:06:35.350229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:32:44.376952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354658+0000","last_change":"2024-06-03T01:04:54.366394+0000","last_active":"2024-06-03T01:06:35.354658+0000","last_peered":"2024-06-03T01:06:35.354658+0000","last_clean":"2024-06-03T01:06:35.354658+0000","last_became_active":"2024-06-03T01:04:54.366067+0000","last_became_peered":"2024-06-03T01:04:54.366067+0000","last_unstale":"2024-06-03T01:06:35.354658+0000","last_undegraded":"2024-06-03T01:06:35.354658+0000","last_fullsized":"2024-06-03T01:06:35.354658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:18:03.772654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75036,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":417,"num_read_kb":293,"num_write":253,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":225108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":575353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32992,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712160,"statfs":{"total":95995035648,"available":95961251840,"internally_reserved":0,"allocated":1859584,"data_stored":1229111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":578877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":575047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1777664,"data_stored":1172045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1159168,"data_stored":570675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1232896,"data_stored":639149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738471,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1183744,"data_stored":572131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1253376,"data_stored":639759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":569993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":574359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32920,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712232,"statfs":{"total":95995035648,"available":95961325568,"internally_reserved":0,"allocated":1785856,"data_stored":1167237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T01:06:59.957 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-03T01:06:59.957 INFO:tasks.cephadm:Setting up client nodes... 2024-06-03T01:06:59.957 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-03T01:07:00.439 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:00.584 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:00 smithi094 ceph-mon[32119]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:07:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:00 smithi179 ceph-mon[37041]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:07:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:00 smithi107 ceph-mon[36622]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:07:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:01 smithi094 ceph-mon[32119]: pgmap v403: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-06-03T01:07:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:01 smithi179 ceph-mon[37041]: pgmap v403: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-06-03T01:07:01.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:01 smithi107 ceph-mon[36622]: pgmap v403: 129 pgs: 129 active+clean; 650 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.852+0000 7fb389c00700 1 -- 172.21.15.94:0/140405154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384105930 msgr2=0x7fb384105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.852+0000 7fb389c00700 1 --2- 172.21.15.94:0/140405154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384105930 0x7fb384105d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb370009920 tx=0x7fb370009c30 comp rx=0 tx=0).stop 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.852+0000 7fb389c00700 1 -- 172.21.15.94:0/140405154 shutdown_connections 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.852+0000 7fb389c00700 1 --2- 172.21.15.94:0/140405154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3841062d0 0x7fb38410cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.852+0000 7fb389c00700 1 --2- 172.21.15.94:0/140405154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb384104730 0x7fb384104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.852+0000 7fb389c00700 1 --2- 172.21.15.94:0/140405154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384105930 0x7fb384105d90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.852+0000 7fb389c00700 1 -- 172.21.15.94:0/140405154 >> 172.21.15.94:0/140405154 conn(0x7fb3840ffcf0 msgr2=0x7fb384102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.853+0000 7fb389c00700 1 -- 172.21.15.94:0/140405154 shutdown_connections 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.853+0000 7fb389c00700 1 -- 172.21.15.94:0/140405154 wait complete. 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 Processor -- start 2024-06-03T01:07:02.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 -- start start 2024-06-03T01:07:02.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384104730 0x7fb38419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:02.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb384105930 0x7fb38419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:02.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3841062d0 0x7fb3841a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:02.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb384110360 con 0x7fb384105930 2024-06-03T01:07:02.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3841101e0 con 0x7fb384104730 2024-06-03T01:07:02.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.854+0000 7fb389c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3841104e0 con 0x7fb3841062d0 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384104730 0x7fb38419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384104730 0x7fb38419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51150/0 (socket says 172.21.15.94:51150) 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb384105930 0x7fb38419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 -- 172.21.15.94:0/3872222410 learned_addr learned my addr 172.21.15.94:0/3872222410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb383400700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3841062d0 0x7fb3841a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 -- 172.21.15.94:0/3872222410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3841062d0 msgr2=0x7fb3841a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3841062d0 0x7fb3841a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 -- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb384105930 msgr2=0x7fb38419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb384105930 0x7fb38419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 -- 172.21.15.94:0/3872222410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3841a2580 con 0x7fb384104730 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.855+0000 7fb382a00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384104730 0x7fb38419a920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb37400e950 tx=0x7fb37400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.856+0000 7fb37b400700 1 -- 172.21.15.94:0/3872222410 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374004230 con 0x7fb384104730 2024-06-03T01:07:02.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.856+0000 7fb37b400700 1 -- 172.21.15.94:0/3872222410 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3740043d0 con 0x7fb384104730 2024-06-03T01:07:02.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.856+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3841a2870 con 0x7fb384104730 2024-06-03T01:07:02.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.856+0000 7fb37b400700 1 -- 172.21.15.94:0/3872222410 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374003dc0 con 0x7fb384104730 2024-06-03T01:07:02.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.856+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3841a2db0 con 0x7fb384104730 2024-06-03T01:07:02.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.857+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb384050a00 con 0x7fb384104730 2024-06-03T01:07:02.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.859+0000 7fb37b400700 1 -- 172.21.15.94:0/3872222410 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb37400c7e0 con 0x7fb384104730 2024-06-03T01:07:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.860+0000 7fb37b400700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3680a2de0 0x7fb3680a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.860+0000 7fb37b400700 1 -- 172.21.15.94:0/3872222410 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb3740cf790 con 0x7fb384104730 2024-06-03T01:07:02.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.861+0000 7fb382000700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3680a2de0 0x7fb3680a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:02.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.861+0000 7fb382000700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3680a2de0 0x7fb3680a52a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb37000fca0 tx=0x7fb370019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:02.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:02.865+0000 7fb37b400700 1 -- 172.21.15.94:0/3872222410 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb37409c1b0 con 0x7fb384104730 2024-06-03T01:07:03.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.182+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb3840630a0 con 0x7fb384104730 2024-06-03T01:07:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.191+0000 7fb37b400700 1 -- 172.21.15.94:0/3872222410 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb37409bab0 con 0x7fb384104730 2024-06-03T01:07:03.192 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-06-03T01:07:03.192 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQA3F11mcSQQCxAATHFU3fR2FFrtRIGnpXQzhA== 2024-06-03T01:07:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3680a2de0 msgr2=0x7fb3680a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3680a2de0 0x7fb3680a52a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb37000fca0 tx=0x7fb370019040 comp rx=0 tx=0).stop 2024-06-03T01:07:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384104730 msgr2=0x7fb38419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384104730 0x7fb38419a920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb37400e950 tx=0x7fb37400ec60 comp rx=0 tx=0).stop 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 shutdown_connections 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb384104730 0x7fb38419a920 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3680a2de0 0x7fb3680a52a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3841062d0 0x7fb3841a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 --2- 172.21.15.94:0/3872222410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb384105930 0x7fb38419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 >> 172.21.15.94:0/3872222410 conn(0x7fb3840ffcf0 msgr2=0x7fb384108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 shutdown_connections 2024-06-03T01:07:03.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:03.195+0000 7fb389c00700 1 -- 172.21.15.94:0/3872222410 wait complete. 2024-06-03T01:07:03.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:03 smithi094 ceph-mon[32119]: pgmap v404: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-06-03T01:07:03.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:03 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3872222410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:07:03.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:03 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:07:03.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:03 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T01:07:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:03 smithi179 ceph-mon[37041]: pgmap v404: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-06-03T01:07:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:03 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3872222410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:07:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:03 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:07:03.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:03 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T01:07:03.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:03 smithi107 ceph-mon[36622]: pgmap v404: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-06-03T01:07:03.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:03 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3872222410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:07:03.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:03 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-03T01:07:03.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:03 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-03T01:07:04.082 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T01:07:04.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-03T01:07:04.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-03T01:07:04.193 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-03T01:07:04.193 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-03T01:07:04.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-03T01:07:04.692 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:05.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:05 smithi094 ceph-mon[32119]: pgmap v405: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:07:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:05 smithi179 ceph-mon[37041]: pgmap v405: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:07:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:05 smithi107 ceph-mon[36622]: pgmap v405: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:07:07.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.105+0000 7feafe200700 1 -- 172.21.15.94:0/1263395572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 msgr2=0x7feaf810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.105+0000 7feafe200700 1 --2- 172.21.15.94:0/1263395572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 0x7feaf810f3c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feaf000b0d0 tx=0x7feaf000b3e0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.106+0000 7feafe200700 1 -- 172.21.15.94:0/1263395572 shutdown_connections 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.106+0000 7feafe200700 1 --2- 172.21.15.94:0/1263395572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 0x7feaf810f3c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.106+0000 7feafe200700 1 --2- 172.21.15.94:0/1263395572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf8102740 0x7feaf8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.106+0000 7feafe200700 1 --2- 172.21.15.94:0/1263395572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81030f0 0x7feaf8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.106+0000 7feafe200700 1 -- 172.21.15.94:0/1263395572 >> 172.21.15.94:0/1263395572 conn(0x7feaf80795f0 msgr2=0x7feaf8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.106+0000 7feafe200700 1 -- 172.21.15.94:0/1263395572 shutdown_connections 2024-06-03T01:07:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.106+0000 7feafe200700 1 -- 172.21.15.94:0/1263395572 wait complete. 2024-06-03T01:07:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.107+0000 7feafe200700 1 Processor -- start 2024-06-03T01:07:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.107+0000 7feafe200700 1 -- start start 2024-06-03T01:07:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feafe200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf8102740 0x7feaf819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feafe200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf81030f0 0x7feaf819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feafe200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 0x7feaf819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feafe200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112780 con 0x7feaf8107840 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feafe200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112600 con 0x7feaf8102740 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feafe200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112900 con 0x7feaf81030f0 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feaefe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf81030f0 0x7feaf819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.108+0000 7feaf7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 0x7feaf819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaf7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf8102740 0x7feaf819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf81030f0 0x7feaf819b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:47852/0 (socket says 172.21.15.94:47852) 2024-06-03T01:07:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 -- 172.21.15.94:0/3093214836 learned_addr learned my addr 172.21.15.94:0/3093214836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 -- 172.21.15.94:0/3093214836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf8102740 msgr2=0x7feaf819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf8102740 0x7feaf819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 -- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 msgr2=0x7feaf819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 0x7feaf819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 -- 172.21.15.94:0/3093214836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf819fd60 con 0x7feaf81030f0 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaf7e00700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 0x7feaf819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaf7400700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf8102740 0x7feaf819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.109+0000 7feaefe00700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf81030f0 0x7feaf819b290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feae000eb30 tx=0x7feae000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.110+0000 7feaf4c00700 1 -- 172.21.15.94:0/3093214836 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae000cbd0 con 0x7feaf81030f0 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.110+0000 7feaf4c00700 1 -- 172.21.15.94:0/3093214836 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0004500 con 0x7feaf81030f0 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.110+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf81a0050 con 0x7feaf81030f0 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.110+0000 7feaf4c00700 1 -- 172.21.15.94:0/3093214836 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0010430 con 0x7feaf81030f0 2024-06-03T01:07:07.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.110+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81a78d0 con 0x7feaf81030f0 2024-06-03T01:07:07.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.111+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf8069180 con 0x7feaf81030f0 2024-06-03T01:07:07.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.115+0000 7feaf4c00700 1 -- 172.21.15.94:0/3093214836 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feae0003f00 con 0x7feaf81030f0 2024-06-03T01:07:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.116+0000 7feaf4c00700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7feadc0a2d30 0x7feadc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.116+0000 7feaf4c00700 1 -- 172.21.15.94:0/3093214836 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7feae00cf890 con 0x7feaf81030f0 2024-06-03T01:07:07.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.116+0000 7feaf7400700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7feadc0a2d30 0x7feadc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:07.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.117+0000 7feaf7400700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7feadc0a2d30 0x7feadc0a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feae8011490 tx=0x7feae8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:07.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.121+0000 7feaf4c00700 1 -- 172.21.15.94:0/3093214836 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feae009c2b0 con 0x7feaf81030f0 2024-06-03T01:07:07.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.412+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7feaf819c0b0 con 0x7feaf81030f0 2024-06-03T01:07:07.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.417+0000 7feaf4c00700 1 -- 172.21.15.94:0/3093214836 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263199 (secure 0 0 0) 0x7feaf819c0b0 con 0x7feaf81030f0 2024-06-03T01:07:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:07:07.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7feadc0a2d30 msgr2=0x7feadc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7feadc0a2d30 0x7feadc0a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feae8011490 tx=0x7feae8009260 comp rx=0 tx=0).stop 2024-06-03T01:07:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf81030f0 msgr2=0x7feaf819b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf81030f0 0x7feaf819b290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feae000eb30 tx=0x7feae000ee40 comp rx=0 tx=0).stop 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 shutdown_connections 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf8102740 0x7feaf819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7feadc0a2d30 0x7feadc0a51f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feaf8107840 0x7feaf819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 --2- 172.21.15.94:0/3093214836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feaf81030f0 0x7feaf819b290 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 >> 172.21.15.94:0/3093214836 conn(0x7feaf80795f0 msgr2=0x7feaf8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 shutdown_connections 2024-06-03T01:07:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:07.423+0000 7feafe200700 1 -- 172.21.15.94:0/3093214836 wait complete. 2024-06-03T01:07:07.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:07 smithi094 ceph-mon[32119]: pgmap v406: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:07.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:07 smithi179 ceph-mon[37041]: pgmap v406: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:07.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:07 smithi107 ceph-mon[36622]: pgmap v406: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:08.362 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":4120992823},{"type":"v1","addr":"172.21.15.94:6801","nonce":4120992823}]},"active_addr":"172.21.15.94:6801/4120992823","active_change":"2024-06-03T00:56:34.112264+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2999518385}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3717514300}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":549573361}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1904227986}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":128190658}]}]} 2024-06-03T01:07:08.367 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-03T01:07:08.367 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-03T01:07:08.367 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:07:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:08 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3093214836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T01:07:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:08 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3093214836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T01:07:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:08 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3093214836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-03T01:07:08.795 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:09 smithi094 ceph-mon[32119]: pgmap v407: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:09 smithi107 ceph-mon[36622]: pgmap v407: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:09.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:09 smithi179 ceph-mon[37041]: pgmap v407: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:11.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 -- 172.21.15.94:0/2583009411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 msgr2=0x7fbdd4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 --2- 172.21.15.94:0/2583009411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 0x7fbdd4108e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009990 tx=0x7fbdc0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 -- 172.21.15.94:0/2583009411 shutdown_connections 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 --2- 172.21.15.94:0/2583009411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd4109390 0x7fbdd4115850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 --2- 172.21.15.94:0/2583009411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd410e980 0x7fbdd410ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 --2- 172.21.15.94:0/2583009411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 0x7fbdd4108e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 -- 172.21.15.94:0/2583009411 >> 172.21.15.94:0/2583009411 conn(0x7fbdd4078200 msgr2=0x7fbdd41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 -- 172.21.15.94:0/2583009411 shutdown_connections 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.184+0000 7fbdd3200700 1 -- 172.21.15.94:0/2583009411 wait complete. 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.185+0000 7fbdd3200700 1 Processor -- start 2024-06-03T01:07:11.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd3200700 1 -- start start 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd3200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 0x7fbdd419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd3200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd4109390 0x7fbdd419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd3200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd410e980 0x7fbdd41a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd3200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd4118d10 con 0x7fbdd41089d0 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd3200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd4118b90 con 0x7fbdd410e980 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd3200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd4118e90 con 0x7fbdd4109390 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 0x7fbdd419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd1400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd4109390 0x7fbdd419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd1400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd4109390 0x7fbdd419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:47874/0 (socket says 172.21.15.94:47874) 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd1400700 1 -- 172.21.15.94:0/250387622 learned_addr learned my addr 172.21.15.94:0/250387622 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:11.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 0x7fbdd419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45418/0 (socket says 172.21.15.94:45418) 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.186+0000 7fbdd2800700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd410e980 0x7fbdd41a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.187+0000 7fbdd1400700 1 -- 172.21.15.94:0/250387622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd410e980 msgr2=0x7fbdd41a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.187+0000 7fbdd1400700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd410e980 0x7fbdd41a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.187+0000 7fbdd1400700 1 -- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 msgr2=0x7fbdd419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.187+0000 7fbdd1400700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 0x7fbdd419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.187+0000 7fbdd1400700 1 -- 172.21.15.94:0/250387622 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd41a40f0 con 0x7fbdd4109390 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.187+0000 7fbdd1400700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd4109390 0x7fbdd419f620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.187+0000 7fbdcaa00700 1 -- 172.21.15.94:0/250387622 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0027070 con 0x7fbdd4109390 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.188+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd41a4380 con 0x7fbdd4109390 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.188+0000 7fbdcaa00700 1 -- 172.21.15.94:0/250387622 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc0003d70 con 0x7fbdd4109390 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.188+0000 7fbdcaa00700 1 -- 172.21.15.94:0/250387622 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000f460 con 0x7fbdd4109390 2024-06-03T01:07:11.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.188+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd41abc60 con 0x7fbdd4109390 2024-06-03T01:07:11.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.189+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd4069180 con 0x7fbdd4109390 2024-06-03T01:07:11.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.191+0000 7fbdcaa00700 1 -- 172.21.15.94:0/250387622 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbdc002a070 con 0x7fbdd4109390 2024-06-03T01:07:11.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.192+0000 7fbdcaa00700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbdb80a2cb0 0x7fbdb80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:11.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.192+0000 7fbdcaa00700 1 -- 172.21.15.94:0/250387622 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbdc00d0870 con 0x7fbdd4109390 2024-06-03T01:07:11.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.193+0000 7fbdd1e00700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbdb80a2cb0 0x7fbdb80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:11.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.193+0000 7fbdd1e00700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbdb80a2cb0 0x7fbdb80a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009be0 tx=0x7fbdc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.195+0000 7fbdcaa00700 1 -- 172.21.15.94:0/250387622 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbdc0031070 con 0x7fbdd4109390 2024-06-03T01:07:11.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.423+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdd402dca0 con 0x7fbdd4109390 2024-06-03T01:07:11.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.425+0000 7fbdcaa00700 1 -- 172.21.15.94:0/250387622 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24723 (secure 0 0 0) 0x7fbdc0087020 con 0x7fbdd4109390 2024-06-03T01:07:11.426 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:11 smithi094 ceph-mon[32119]: pgmap v408: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:07:11.427 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":100,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:06:35.341979+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:07:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.429+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbdb80a2cb0 msgr2=0x7fbdb80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.429+0000 7fbdd3200700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbdb80a2cb0 0x7fbdb80a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009be0 tx=0x7fbdc4009260 comp rx=0 tx=0).stop 2024-06-03T01:07:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.429+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd4109390 msgr2=0x7fbdd419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.429+0000 7fbdd3200700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd4109390 0x7fbdd419f620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc000bab0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 shutdown_connections 2024-06-03T01:07:11.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd410e980 0x7fbdd41a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbdb80a2cb0 0x7fbdb80a5170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbdd41089d0 0x7fbdd419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 --2- 172.21.15.94:0/250387622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbdd4109390 0x7fbdd419f620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 >> 172.21.15.94:0/250387622 conn(0x7fbdd4078200 msgr2=0x7fbdd4078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 shutdown_connections 2024-06-03T01:07:11.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:11.430+0000 7fbdd3200700 1 -- 172.21.15.94:0/250387622 wait complete. 2024-06-03T01:07:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:11 smithi107 ceph-mon[36622]: pgmap v408: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:11 smithi179 ceph-mon[37041]: pgmap v408: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:12.351 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-03T01:07:12.352 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:07:12.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:12 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/250387622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:07:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:12 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/250387622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:07:12.793 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:12 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/250387622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:07:13.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:13 smithi094 ceph-mon[32119]: pgmap v409: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:13.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:13 smithi107 ceph-mon[36622]: pgmap v409: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:13.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:13 smithi179 ceph-mon[37041]: pgmap v409: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-03T01:07:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:14 smithi094 ceph-mon[32119]: pgmap v410: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:07:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:14 smithi107 ceph-mon[36622]: pgmap v410: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:07:14.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:14 smithi179 ceph-mon[37041]: pgmap v410: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:07:15.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.183+0000 7fdc6ec00700 1 -- 172.21.15.94:0/707453235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102fd0 msgr2=0x7fdc68107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:15.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.183+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/707453235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102fd0 0x7fdc68107020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009990 tx=0x7fdc54009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.184+0000 7fdc6ec00700 1 -- 172.21.15.94:0/707453235 shutdown_connections 2024-06-03T01:07:15.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.184+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/707453235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc68107700 0x7fdc6810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.184+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/707453235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68102620 0x7fdc68102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.184+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/707453235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102fd0 0x7fdc68107020 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.184+0000 7fdc6ec00700 1 -- 172.21.15.94:0/707453235 >> 172.21.15.94:0/707453235 conn(0x7fdc680fdf10 msgr2=0x7fdc68100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:15.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.184+0000 7fdc6ec00700 1 -- 172.21.15.94:0/707453235 shutdown_connections 2024-06-03T01:07:15.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.184+0000 7fdc6ec00700 1 -- 172.21.15.94:0/707453235 wait complete. 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 Processor -- start 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 -- start start 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102620 0x7fdc6819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc68102fd0 0x7fdc6819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68107700 0x7fdc6819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68112710 con 0x7fdc68102fd0 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68112590 con 0x7fdc68102620 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.185+0000 7fdc6ec00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68112890 con 0x7fdc68107700 2024-06-03T01:07:15.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6ce00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc68102fd0 0x7fdc6819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6ce00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc68102fd0 0x7fdc6819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54144/0 (socket says 172.21.15.94:54144) 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6ce00700 1 -- 172.21.15.94:0/2273933755 learned_addr learned my addr 172.21.15.94:0/2273933755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6e200700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68107700 0x7fdc6819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6d800700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102620 0x7fdc6819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6e200700 1 -- 172.21.15.94:0/2273933755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102620 msgr2=0x7fdc6819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6e200700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102620 0x7fdc6819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6e200700 1 -- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc68102fd0 msgr2=0x7fdc6819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6e200700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc68102fd0 0x7fdc6819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6e200700 1 -- 172.21.15.94:0/2273933755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6819fdb0 con 0x7fdc68107700 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.186+0000 7fdc6e200700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68107700 0x7fdc6819f670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e970 tx=0x7fdc6000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.187+0000 7fdc66000700 1 -- 172.21.15.94:0/2273933755 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60004220 con 0x7fdc68107700 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.187+0000 7fdc66000700 1 -- 172.21.15.94:0/2273933755 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc60004d10 con 0x7fdc68107700 2024-06-03T01:07:15.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.187+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc681a00a0 con 0x7fdc68107700 2024-06-03T01:07:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.187+0000 7fdc66000700 1 -- 172.21.15.94:0/2273933755 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60010700 con 0x7fdc68107700 2024-06-03T01:07:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.187+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc681a7920 con 0x7fdc68107700 2024-06-03T01:07:15.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.188+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc68069180 con 0x7fdc68107700 2024-06-03T01:07:15.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.190+0000 7fdc66000700 1 -- 172.21.15.94:0/2273933755 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdc60004750 con 0x7fdc68107700 2024-06-03T01:07:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.191+0000 7fdc66000700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdc500a2d30 0x7fdc500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:15.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.191+0000 7fdc66000700 1 -- 172.21.15.94:0/2273933755 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdc600cf350 con 0x7fdc68107700 2024-06-03T01:07:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.192+0000 7fdc6d800700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdc500a2d30 0x7fdc500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:15.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.192+0000 7fdc6d800700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdc500a2d30 0x7fdc500a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00f540 tx=0x7fdc5c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:15.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.193+0000 7fdc66000700 1 -- 172.21.15.94:0/2273933755 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdc6009be00 con 0x7fdc68107700 2024-06-03T01:07:15.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.418+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc6819c100 con 0x7fdc68107700 2024-06-03T01:07:15.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.420+0000 7fdc66000700 1 -- 172.21.15.94:0/2273933755 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24723 (secure 0 0 0) 0x7fdc6819c100 con 0x7fdc68107700 2024-06-03T01:07:15.421 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:07:15.421 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":100,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:06:35.341979+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdc500a2d30 msgr2=0x7fdc500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdc500a2d30 0x7fdc500a51f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00f540 tx=0x7fdc5c00b040 comp rx=0 tx=0).stop 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68107700 msgr2=0x7fdc6819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68107700 0x7fdc6819f670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e970 tx=0x7fdc6000ec80 comp rx=0 tx=0).stop 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 shutdown_connections 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc68102620 0x7fdc6819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fdc500a2d30 0x7fdc500a51f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc68107700 0x7fdc6819f670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 --2- 172.21.15.94:0/2273933755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc68102fd0 0x7fdc6819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:15.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.423+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 >> 172.21.15.94:0/2273933755 conn(0x7fdc680fdf10 msgr2=0x7fdc6810da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:15.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.424+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 shutdown_connections 2024-06-03T01:07:15.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:15.424+0000 7fdc6ec00700 1 -- 172.21.15.94:0/2273933755 wait complete. 2024-06-03T01:07:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:15 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2273933755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:07:15.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:15 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2273933755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:07:15.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:15 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2273933755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:07:16.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-03T01:07:16.317 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-03T01:07:16.317 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-03T01:07:16.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-03T01:07:16.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-03T01:07:16.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-03T01:07:16.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-03T01:07:16.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-03T01:07:16.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-03T01:07:16.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-03T01:07:16.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-03T01:07:16.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-03T01:07:16.504 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:16 smithi094 ceph-mon[32119]: pgmap v411: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:07:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:16 smithi107 ceph-mon[36622]: pgmap v411: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:07:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:16 smithi179 ceph-mon[37041]: pgmap v411: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:07:17.353 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.359 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.361 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.364 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.368 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.826 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.837 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.838 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.838 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.840 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.842 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:17.848 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:18.681 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:18 smithi094 ceph-mon[32119]: pgmap v412: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:18 smithi107 ceph-mon[36622]: pgmap v412: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:18 smithi179 ceph-mon[37041]: pgmap v412: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:20.540 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:20 smithi094 ceph-mon[32119]: pgmap v413: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:20.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:20 smithi107 ceph-mon[36622]: pgmap v413: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:20 smithi179 ceph-mon[37041]: pgmap v413: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:22.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:22 smithi094 ceph-mon[32119]: pgmap v414: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:22 smithi107 ceph-mon[36622]: pgmap v414: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:22.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:22 smithi179 ceph-mon[37041]: pgmap v414: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:24 smithi107 ceph-mon[36622]: pgmap v415: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:07:24.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:24 smithi094 ceph-mon[32119]: pgmap v415: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:07:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:24 smithi179 ceph-mon[37041]: pgmap v415: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:07:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:26 smithi094 ceph-mon[32119]: pgmap v416: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:07:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:26 smithi107 ceph-mon[36622]: pgmap v416: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:07:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:26 smithi179 ceph-mon[37041]: pgmap v416: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:07:29.405 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:29 smithi094 ceph-mon[32119]: pgmap v417: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:29 smithi107 ceph-mon[36622]: pgmap v417: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:29 smithi179 ceph-mon[37041]: pgmap v417: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 -- 172.21.15.94:0/1151023897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b4103fc0 msgr2=0x7ff5b4104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 --2- 172.21.15.94:0/1151023897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b4103fc0 0x7ff5b4104440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00099c0 tx=0x7ff5a0009cd0 comp rx=0 tx=0).stop 2024-06-03T01:07:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 -- 172.21.15.94:0/1151023897 shutdown_connections 2024-06-03T01:07:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 --2- 172.21.15.94:0/1151023897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b41026d0 0x7ff5b4102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 --2- 172.21.15.94:0/1151023897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4104980 0x7ff5b410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 --2- 172.21.15.94:0/1151023897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b4103fc0 0x7ff5b4104440 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:29.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 -- 172.21.15.94:0/1151023897 >> 172.21.15.94:0/1151023897 conn(0x7ff5b40fde80 msgr2=0x7ff5b41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 -- 172.21.15.94:0/1151023897 shutdown_connections 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.164+0000 7ff5bac00700 1 -- 172.21.15.94:0/1151023897 wait complete. 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.165+0000 7ff5bac00700 1 Processor -- start 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5bac00700 1 -- start start 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5bac00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b41026d0 0x7ff5b419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5bac00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b4103fc0 0x7ff5b419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5bac00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4104980 0x7ff5b419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5bac00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41127a0 con 0x7ff5b4104980 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5bac00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b4112620 con 0x7ff5b4103fc0 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5bac00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b4112920 con 0x7ff5b41026d0 2024-06-03T01:07:29.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5ba200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4104980 0x7ff5b419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5ba200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4104980 0x7ff5b419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58578/0 (socket says 172.21.15.94:58578) 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5ba200700 1 -- 172.21.15.94:0/3161969608 learned_addr learned my addr 172.21.15.94:0/3161969608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.166+0000 7ff5b8e00700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b4103fc0 0x7ff5b419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5b9800700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b41026d0 0x7ff5b419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5ba200700 1 -- 172.21.15.94:0/3161969608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b41026d0 msgr2=0x7ff5b419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5ba200700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5b41026d0 0x7ff5b419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5ba200700 1 -- 172.21.15.94:0/3161969608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b4103fc0 msgr2=0x7ff5b419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5ba200700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b4103fc0 0x7ff5b419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5ba200700 1 -- 172.21.15.94:0/3161969608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b419fd80 con 0x7ff5b4104980 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5b8e00700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b4103fc0 0x7ff5b419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:29.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5ba200700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4104980 0x7ff5b419f640 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c370 tx=0x7ff5ac00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00ce20 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ac0090d0 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.167+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b41a0010 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.168+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac007500 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.168+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b41a78f0 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.169+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff5b40ff5c0 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.171+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff5ac01a020 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.172+0000 7ff5b2000700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff59c0a2cb0 0x7ff59c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.172+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff5ac0cfa80 con 0x7ff5b4104980 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.172+0000 7ff5b9800700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff59c0a2cb0 0x7ff59c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.173+0000 7ff5b2000700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] conn(0x7ff59c0ade90 0x7ff59c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.173+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 --> [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5ac00be50 con 0x7ff59c0ade90 2024-06-03T01:07:29.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.173+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff5ac0cfe30 con 0x7ff5b4104980 2024-06-03T01:07:29.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.173+0000 7ff5b8e00700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] conn(0x7ff59c0ade90 0x7ff59c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:29.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.173+0000 7ff5b9800700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff59c0a2cb0 0x7ff59c0a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009bf0 tx=0x7ff5a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:29.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.174+0000 7ff5b8e00700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] conn(0x7ff59c0ade90 0x7ff59c0b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:29.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.175+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== osd.3 v2:172.21.15.94:6810/667050825 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5ac00be50 con 0x7ff59c0ade90 2024-06-03T01:07:29.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.193+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 --> [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff5b4103ad0 con 0x7ff59c0ade90 2024-06-03T01:07:29.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5b2000700 1 -- 172.21.15.94:0/3161969608 <== osd.3 v2:172.21.15.94:6810/667050825 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff5b4103ad0 con 0x7ff59c0ade90 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] conn(0x7ff59c0ade90 msgr2=0x7ff59c0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5bac00700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] conn(0x7ff59c0ade90 0x7ff59c0b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff59c0a2cb0 msgr2=0x7ff59c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5bac00700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff59c0a2cb0 0x7ff59c0a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009bf0 tx=0x7ff5a8009260 comp rx=0 tx=0).stop 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4104980 msgr2=0x7ff5b419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5bac00700 1 --2- 172.21.15.94:0/3161969608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5b4104980 0x7ff5b419f640 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c370 tx=0x7ff5ac00c680 comp rx=0 tx=0).stop 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.194+0000 7ff5ba200700 1 -- 172.21.15.94:0/3161969608 reap_dead start 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.195+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 shutdown_connections 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.195+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 >> 172.21.15.94:0/3161969608 conn(0x7ff5b40fde80 msgr2=0x7ff5b410db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.195+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 shutdown_connections 2024-06-03T01:07:29.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:28.195+0000 7ff5bac00700 1 -- 172.21.15.94:0/3161969608 wait complete. 2024-06-03T01:07:30.907 INFO:teuthology.orchestra.run.smithi094.stdout:107374182493 2024-06-03T01:07:30.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-03T01:07:31.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:31 smithi094 ceph-mon[32119]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:31 smithi107 ceph-mon[36622]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:31.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:31 smithi179 ceph-mon[37041]: pgmap v418: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:33.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 -- 172.21.15.94:0/2620172758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1ac107450 msgr2=0x7fa1ac107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 --2- 172.21.15.94:0/2620172758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1ac107450 0x7fa1ac107830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009990 tx=0x7fa1a0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 -- 172.21.15.94:0/2620172758 shutdown_connections 2024-06-03T01:07:33.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 --2- 172.21.15.94:0/2620172758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ac102ee0 0x7fa1ac10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 --2- 172.21.15.94:0/2620172758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1ac107450 0x7fa1ac107830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 --2- 172.21.15.94:0/2620172758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1ac102540 0x7fa1ac1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 -- 172.21.15.94:0/2620172758 >> 172.21.15.94:0/2620172758 conn(0x7fa1ac0795f0 msgr2=0x7fa1ac079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 -- 172.21.15.94:0/2620172758 shutdown_connections 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.352+0000 7fa1b3c00700 1 -- 172.21.15.94:0/2620172758 wait complete. 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.354+0000 7fa1b3c00700 1 Processor -- start 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.354+0000 7fa1b3c00700 1 -- start start 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b3c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1ac102540 0x7fa1ac19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b3c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ac102ee0 0x7fa1ac19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b3c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1ac107450 0x7fa1ac198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b3c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac1127c0 con 0x7fa1ac102540 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b3c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac112640 con 0x7fa1ac107450 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b3c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac112940 con 0x7fa1ac102ee0 2024-06-03T01:07:33.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b1200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1ac102540 0x7fa1ac19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ac102ee0 0x7fa1ac19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1b1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1ac107450 0x7fa1ac198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ac102ee0 0x7fa1ac19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:37632/0 (socket says 172.21.15.94:37632) 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 -- 172.21.15.94:0/986093576 learned_addr learned my addr 172.21.15.94:0/986093576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 -- 172.21.15.94:0/986093576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1ac107450 msgr2=0x7fa1ac198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1ac107450 0x7fa1ac198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 -- 172.21.15.94:0/986093576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1ac102540 msgr2=0x7fa1ac19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1ac102540 0x7fa1ac19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.355+0000 7fa1abe00700 1 -- 172.21.15.94:0/986093576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac199220 con 0x7fa1ac102ee0 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.356+0000 7fa1abe00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ac102ee0 0x7fa1ac19edf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa19800eb30 tx=0x7fa19800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.356+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19800cbd0 con 0x7fa1ac102ee0 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.356+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa198004500 con 0x7fa1ac102ee0 2024-06-03T01:07:33.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.356+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198010430 con 0x7fa1ac102ee0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.356+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac199510 con 0x7fa1ac102ee0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.357+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac1a5780 con 0x7fa1ac102ee0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.357+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1ac062c20 con 0x7fa1ac102ee0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.360+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa198003f00 con 0x7fa1ac102ee0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.362+0000 7fa1a9600700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1940a2d30 0x7fa1940a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.363+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa1980cfb50 con 0x7fa1ac102ee0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.363+0000 7fa1b1200700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1940a2d30 0x7fa1940a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.363+0000 7fa1a9600700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] conn(0x7fa1940ade40 0x7fa1940b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.363+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 --> [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa19801aec0 con 0x7fa1940ade40 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.363+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa1980cff00 con 0x7fa1ac102ee0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.363+0000 7fa1b1c00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] conn(0x7fa1940ade40 0x7fa1940b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.364+0000 7fa1b1200700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1940a2d30 0x7fa1940a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000b2e0 tx=0x7fa1a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:33.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.364+0000 7fa1b1c00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] conn(0x7fa1940ade40 0x7fa1940b02e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.365+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== osd.10 v2:172.21.15.107:6824/1849339133 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa19801aec0 con 0x7fa1940ade40 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.383+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 --> [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1ac0630f0 con 0x7fa1940ade40 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.384+0000 7fa1a9600700 1 -- 172.21.15.94:0/986093576 <== osd.10 v2:172.21.15.107:6824/1849339133 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa1ac0630f0 con 0x7fa1940ade40 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.384+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 >> [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] conn(0x7fa1940ade40 msgr2=0x7fa1940b02e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.384+0000 7fa1b3c00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] conn(0x7fa1940ade40 0x7fa1940b02e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1940a2d30 msgr2=0x7fa1940a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1940a2d30 0x7fa1940a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000b2e0 tx=0x7fa1a0005960 comp rx=0 tx=0).stop 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ac102ee0 msgr2=0x7fa1ac19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 --2- 172.21.15.94:0/986093576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1ac102ee0 0x7fa1ac19edf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa19800eb30 tx=0x7fa19800ee40 comp rx=0 tx=0).stop 2024-06-03T01:07:33.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b1c00700 1 -- 172.21.15.94:0/986093576 reap_dead start 2024-06-03T01:07:33.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 shutdown_connections 2024-06-03T01:07:33.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 >> 172.21.15.94:0/986093576 conn(0x7fa1ac0795f0 msgr2=0x7fa1ac100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:33.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 shutdown_connections 2024-06-03T01:07:33.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.385+0000 7fa1b3c00700 1 -- 172.21.15.94:0/986093576 wait complete. 2024-06-03T01:07:33.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:33 smithi094 ceph-mon[32119]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:33.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.464+0000 7fa19ea00700 1 -- 172.21.15.94:0/1454496477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1981030f0 msgr2=0x7fa198107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.464+0000 7fa19ea00700 1 --2- 172.21.15.94:0/1454496477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1981030f0 0x7fa198107110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa184009990 tx=0x7fa184009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.464+0000 7fa19ea00700 1 -- 172.21.15.94:0/1454496477 shutdown_connections 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.464+0000 7fa19ea00700 1 --2- 172.21.15.94:0/1454496477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198102740 0x7fa198102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.464+0000 7fa19ea00700 1 --2- 172.21.15.94:0/1454496477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa198107840 0x7fa19810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.464+0000 7fa19ea00700 1 --2- 172.21.15.94:0/1454496477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1981030f0 0x7fa198107110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.464+0000 7fa19ea00700 1 -- 172.21.15.94:0/1454496477 >> 172.21.15.94:0/1454496477 conn(0x7fa1980795f0 msgr2=0x7fa198079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.465+0000 7fa19ea00700 1 -- 172.21.15.94:0/1454496477 shutdown_connections 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.465+0000 7fa19ea00700 1 -- 172.21.15.94:0/1454496477 wait complete. 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.465+0000 7fa19ea00700 1 Processor -- start 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ea00700 1 -- start start 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198102740 0x7fa19819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1981030f0 0x7fa19819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ea00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa198107840 0x7fa19819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ea00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa198112750 con 0x7fa1981030f0 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ea00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981125d0 con 0x7fa198102740 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ea00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981128d0 con 0x7fa198107840 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa197400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1981030f0 0x7fa19819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa197400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1981030f0 0x7fa19819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58626/0 (socket says 172.21.15.94:58626) 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa197400700 1 -- 172.21.15.94:0/1768382330 learned_addr learned my addr 172.21.15.94:0/1768382330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa19ca00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa198107840 0x7fa19819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.466+0000 7fa197e00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198102740 0x7fa19819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa19ca00700 1 -- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198102740 msgr2=0x7fa19819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa19ca00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa198102740 0x7fa19819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa19ca00700 1 -- 172.21.15.94:0/1768382330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1981030f0 msgr2=0x7fa19819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa19ca00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1981030f0 0x7fa19819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa19ca00700 1 -- 172.21.15.94:0/1768382330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19819fdb0 con 0x7fa198107840 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa19ca00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa198107840 0x7fa19819f670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa19000efc0 tx=0x7fa19000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900092e0 con 0x7fa198107840 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.467+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1981a0040 con 0x7fa198107840 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.468+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa190004bb0 con 0x7fa198107840 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.468+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190004280 con 0x7fa198107840 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.468+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1981a7920 con 0x7fa198107840 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.468+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa19810ea70 con 0x7fa198107840 2024-06-03T01:07:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.471+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa19001b030 con 0x7fa198107840 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.472+0000 7fa194c00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa17c0a2d30 0x7fa17c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.472+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa1900cf3f0 con 0x7fa198107840 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.472+0000 7fa194c00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] conn(0x7fa17c0adf10 0x7fa17c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.472+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 --> [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1900020b0 con 0x7fa17c0adf10 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.472+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa1900cf7a0 con 0x7fa198107840 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.472+0000 7fa197e00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa17c0a2d30 0x7fa17c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.473+0000 7fa197400700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] conn(0x7fa17c0adf10 0x7fa17c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.473+0000 7fa197400700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] conn(0x7fa17c0adf10 0x7fa17c0b03b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.473+0000 7fa197e00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa17c0a2d30 0x7fa17c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa188007c60 tx=0x7fa1880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.474+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== osd.4 v2:172.21.15.107:6808/1882504860 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1900020b0 con 0x7fa17c0adf10 2024-06-03T01:07:33.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.492+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 --> [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa198103bd0 con 0x7fa17c0adf10 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.492+0000 7fa194c00700 1 -- 172.21.15.94:0/1768382330 <== osd.4 v2:172.21.15.107:6808/1882504860 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa198103bd0 con 0x7fa17c0adf10 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] conn(0x7fa17c0adf10 msgr2=0x7fa17c0b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] conn(0x7fa17c0adf10 0x7fa17c0b03b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa17c0a2d30 msgr2=0x7fa17c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa17c0a2d30 0x7fa17c0a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa188007c60 tx=0x7fa1880073d0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa198107840 msgr2=0x7fa19819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 --2- 172.21.15.94:0/1768382330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa198107840 0x7fa19819f670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa19000efc0 tx=0x7fa19000c3a0 comp rx=0 tx=0).stop 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ca00700 1 -- 172.21.15.94:0/1768382330 reap_dead start 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 shutdown_connections 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 >> 172.21.15.94:0/1768382330 conn(0x7fa1980795f0 msgr2=0x7fa1981007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 shutdown_connections 2024-06-03T01:07:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.493+0000 7fa19ea00700 1 -- 172.21.15.94:0/1768382330 wait complete. 2024-06-03T01:07:33.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:33 smithi107 ceph-mon[36622]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:33.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:33 smithi179 ceph-mon[37041]: pgmap v419: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:34.617 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:34 smithi094 ceph-mon[32119]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T01:07:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:34 smithi107 ceph-mon[36622]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T01:07:34.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:34 smithi179 ceph-mon[37041]: pgmap v420: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-06-03T01:07:35.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 -- 172.21.15.94:0/954214268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14113660 msgr2=0x7fee14115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 --2- 172.21.15.94:0/954214268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14113660 0x7fee14115a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00b0d0 tx=0x7fee0c00b3e0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 -- 172.21.15.94:0/954214268 shutdown_connections 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 --2- 172.21.15.94:0/954214268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14113660 0x7fee14115a50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 --2- 172.21.15.94:0/954214268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee140746c0 0x7fee14074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 --2- 172.21.15.94:0/954214268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee14074fe0 0x7fee14113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 -- 172.21.15.94:0/954214268 >> 172.21.15.94:0/954214268 conn(0x7fee140795f0 msgr2=0x7fee14079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 -- 172.21.15.94:0/954214268 shutdown_connections 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.684+0000 7fee19600700 1 -- 172.21.15.94:0/954214268 wait complete. 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 Processor -- start 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 -- start start 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee140746c0 0x7fee1419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14074fe0 0x7fee1419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee14113660 0x7fee141a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee14118ea0 con 0x7fee14074fe0 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee14118d20 con 0x7fee14113660 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.685+0000 7fee19600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee14119020 con 0x7fee140746c0 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14074fe0 0x7fee1419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14074fe0 0x7fee1419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58646/0 (socket says 172.21.15.94:58646) 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 -- 172.21.15.94:0/318733427 learned_addr learned my addr 172.21.15.94:0/318733427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee12a00700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee140746c0 0x7fee1419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee13400700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee14113660 0x7fee141a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 -- 172.21.15.94:0/318733427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee140746c0 msgr2=0x7fee1419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee140746c0 0x7fee1419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 -- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee14113660 msgr2=0x7fee141a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee14113660 0x7fee141a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 -- 172.21.15.94:0/318733427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee141a4230 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.686+0000 7fee0be00700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14074fe0 0x7fee1419f7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d980 tx=0x7fedfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.687+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc016070 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.687+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedfc00fd70 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.687+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee141a4520 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.687+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc019360 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.687+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee141abda0 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.688+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee14112cf0 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.690+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fedfc019070 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.691+0000 7fee0b400700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fedf80a2d30 0x7fedf80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.691+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fedfc0cf070 con 0x7fee14074fe0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.692+0000 7fee12a00700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fedf80a2d30 0x7fedf80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.692+0000 7fee0b400700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] conn(0x7fedf80adf10 0x7fedf80b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.692+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 --> [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedf80b0a80 con 0x7fedf80adf10 2024-06-03T01:07:35.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.692+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fedfc0cf420 con 0x7fee14074fe0 2024-06-03T01:07:35.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.692+0000 7fee13400700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] conn(0x7fedf80adf10 0x7fedf80b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.692+0000 7fee12a00700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fedf80a2d30 0x7fedf80a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee04005950 tx=0x7fee0400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:35.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.693+0000 7fee13400700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] conn(0x7fedf80adf10 0x7fedf80b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:35.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.694+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== osd.7 v2:172.21.15.107:6816/2599842283 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedf80b0a80 con 0x7fedf80adf10 2024-06-03T01:07:35.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.711+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 --> [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee14075df0 con 0x7fedf80adf10 2024-06-03T01:07:35.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.712+0000 7fee0b400700 1 -- 172.21.15.94:0/318733427 <== osd.7 v2:172.21.15.107:6816/2599842283 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fee14075df0 con 0x7fedf80adf10 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] conn(0x7fedf80adf10 msgr2=0x7fedf80b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] conn(0x7fedf80adf10 0x7fedf80b03b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fedf80a2d30 msgr2=0x7fedf80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fedf80a2d30 0x7fedf80a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee04005950 tx=0x7fee0400b410 comp rx=0 tx=0).stop 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14074fe0 msgr2=0x7fee1419f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 --2- 172.21.15.94:0/318733427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee14074fe0 0x7fee1419f7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d980 tx=0x7fedfc00dc90 comp rx=0 tx=0).stop 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee13400700 1 -- 172.21.15.94:0/318733427 reap_dead start 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 shutdown_connections 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 >> 172.21.15.94:0/318733427 conn(0x7fee140795f0 msgr2=0x7fee14112350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.713+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 shutdown_connections 2024-06-03T01:07:35.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.714+0000 7fee19600700 1 -- 172.21.15.94:0/318733427 wait complete. 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 -- 172.21.15.94:0/3900086683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c81030f0 msgr2=0x7f79c8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 --2- 172.21.15.94:0/3900086683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c81030f0 0x7f79c8107110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009990 tx=0x7f79b0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 -- 172.21.15.94:0/3900086683 shutdown_connections 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 --2- 172.21.15.94:0/3900086683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c8107840 0x7f79c810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 --2- 172.21.15.94:0/3900086683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79c8102740 0x7f79c8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 --2- 172.21.15.94:0/3900086683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c81030f0 0x7f79c8107110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 -- 172.21.15.94:0/3900086683 >> 172.21.15.94:0/3900086683 conn(0x7f79c80795f0 msgr2=0x7f79c8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.819+0000 7f79ce600700 1 -- 172.21.15.94:0/3900086683 shutdown_connections 2024-06-03T01:07:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.820+0000 7f79ce600700 1 -- 172.21.15.94:0/3900086683 wait complete. 2024-06-03T01:07:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 Processor -- start 2024-06-03T01:07:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 -- start start 2024-06-03T01:07:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c8102740 0x7f79c819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c81030f0 0x7f79c819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79c8107840 0x7f79c819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81127c0 con 0x7f79c8107840 2024-06-03T01:07:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8112640 con 0x7f79c81030f0 2024-06-03T01:07:35.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79ce600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8112940 con 0x7f79c8102740 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79c7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79c8107840 0x7f79c819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.821+0000 7f79c7400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c8102740 0x7f79c819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79c8107840 0x7f79c819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59970/0 (socket says 172.21.15.94:59970) 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7e00700 1 -- 172.21.15.94:0/350343677 learned_addr learned my addr 172.21.15.94:0/350343677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79bea00700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c81030f0 0x7f79c819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7400700 1 -- 172.21.15.94:0/350343677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c81030f0 msgr2=0x7f79c819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7400700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c81030f0 0x7f79c819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7400700 1 -- 172.21.15.94:0/350343677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79c8107840 msgr2=0x7f79c819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7400700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79c8107840 0x7f79c819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7400700 1 -- 172.21.15.94:0/350343677 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c819fe30 con 0x7f79c8102740 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c7400700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c8102740 0x7f79c819ae20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79b800e9f0 tx=0x7f79b800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.822+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b800cb70 con 0x7f79c8102740 2024-06-03T01:07:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.823+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b800ee90 con 0x7f79c8102740 2024-06-03T01:07:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.823+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c81a0120 con 0x7f79c8102740 2024-06-03T01:07:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.823+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8003dc0 con 0x7f79c8102740 2024-06-03T01:07:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.823+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c81a79a0 con 0x7f79c8102740 2024-06-03T01:07:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.823+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79c8075a20 con 0x7f79c8102740 2024-06-03T01:07:35.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.826+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f79b8003f60 con 0x7f79c8102740 2024-06-03T01:07:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.827+0000 7f79c4c00700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f79ac0a2cb0 0x7f79ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.827+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f79b80cf9b0 con 0x7f79c8102740 2024-06-03T01:07:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.828+0000 7f79bea00700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f79ac0a2cb0 0x7f79ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.828+0000 7f79c4c00700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] conn(0x7f79ac0ade90 0x7f79ac0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.828+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 --> [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79b8023e50 con 0x7f79ac0ade90 2024-06-03T01:07:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.828+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f79b80cfd60 con 0x7f79c8102740 2024-06-03T01:07:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.828+0000 7f79bea00700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f79ac0a2cb0 0x7f79ac0a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79c810f770 tx=0x7f79b000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.828+0000 7f79c7e00700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] conn(0x7f79ac0ade90 0x7f79ac0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.829+0000 7f79c7e00700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] conn(0x7f79ac0ade90 0x7f79ac0b0330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.830+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== osd.5 v2:172.21.15.179:6808/1536083025 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79b8023e50 con 0x7f79ac0ade90 2024-06-03T01:07:35.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.848+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 --> [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79c8103bd0 con 0x7f79ac0ade90 2024-06-03T01:07:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.848+0000 7f79c4c00700 1 -- 172.21.15.94:0/350343677 <== osd.5 v2:172.21.15.179:6808/1536083025 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f79c8103bd0 con 0x7f79ac0ade90 2024-06-03T01:07:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.849+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] conn(0x7f79ac0ade90 msgr2=0x7f79ac0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.849+0000 7f79ce600700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] conn(0x7f79ac0ade90 0x7f79ac0b0330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:35.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.849+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f79ac0a2cb0 msgr2=0x7f79ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.849+0000 7f79ce600700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f79ac0a2cb0 0x7f79ac0a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79c810f770 tx=0x7f79b000b260 comp rx=0 tx=0).stop 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.849+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c8102740 msgr2=0x7f79c819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.849+0000 7f79ce600700 1 --2- 172.21.15.94:0/350343677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79c8102740 0x7f79c819ae20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79b800e9f0 tx=0x7f79b800ed00 comp rx=0 tx=0).stop 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.849+0000 7f79c7e00700 1 -- 172.21.15.94:0/350343677 reap_dead start 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.850+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 shutdown_connections 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.850+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 >> 172.21.15.94:0/350343677 conn(0x7f79c80795f0 msgr2=0x7f79c8100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.850+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 shutdown_connections 2024-06-03T01:07:35.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:35.850+0000 7f79ce600700 1 -- 172.21.15.94:0/350343677 wait complete. 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 -- 172.21.15.94:0/1585964785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 msgr2=0x7fe7481088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 --2- 172.21.15.94:0/1585964785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 0x7fe7481088e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe738009990 tx=0x7fe738009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 -- 172.21.15.94:0/1585964785 shutdown_connections 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 --2- 172.21.15.94:0/1585964785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74806ab30 0x7fe74810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 --2- 172.21.15.94:0/1585964785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 0x7fe7481088e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 --2- 172.21.15.94:0/1585964785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe74806a170 0x7fe74806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 -- 172.21.15.94:0/1585964785 >> 172.21.15.94:0/1585964785 conn(0x7fe748078340 msgr2=0x7fe748078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:36.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 -- 172.21.15.94:0/1585964785 shutdown_connections 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.530+0000 7fe74de00700 1 -- 172.21.15.94:0/1585964785 wait complete. 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.531+0000 7fe74de00700 1 Processor -- start 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.531+0000 7fe74de00700 1 -- start start 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.531+0000 7fe74de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe74806a170 0x7fe74819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe74de00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74806ab30 0x7fe74819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe74de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 0x7fe74819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe74de00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7481127d0 con 0x7fe748108500 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe74de00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe748112650 con 0x7fe74806a170 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe74de00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe748112950 con 0x7fe74806ab30 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe747400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 0x7fe74819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:36.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe747400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 0x7fe74819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58652/0 (socket says 172.21.15.94:58652) 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe747400700 1 -- 172.21.15.94:0/3921884257 learned_addr learned my addr 172.21.15.94:0/3921884257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe746a00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe74806a170 0x7fe74819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe746000700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74806ab30 0x7fe74819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe747400700 1 -- 172.21.15.94:0/3921884257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74806ab30 msgr2=0x7fe74819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe747400700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74806ab30 0x7fe74819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe747400700 1 -- 172.21.15.94:0/3921884257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe74806a170 msgr2=0x7fe74819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.532+0000 7fe747400700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe74806a170 0x7fe74819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe747400700 1 -- 172.21.15.94:0/3921884257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74819fe70 con 0x7fe748108500 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe746000700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe74806ab30 0x7fe74819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe746a00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe74806a170 0x7fe74819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe747400700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 0x7fe74819f730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe74000ef30 tx=0x7fe74000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400092e0 con 0x7fe748108500 2024-06-03T01:07:36.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe740004bb0 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7481a0100 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7481a79e0 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.533+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740004280 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.534+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe74810e0d0 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.536+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe74001b030 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.538+0000 7fe74ca00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe72c0a2cb0 0x7fe72c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.538+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe7400cf940 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.538+0000 7fe746a00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe72c0a2cb0 0x7fe72c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.538+0000 7fe74ca00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] conn(0x7fe72c0ade90 0x7fe72c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.538+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 --> [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe74001fe50 con 0x7fe72c0ade90 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.538+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fe7400cfcf0 con 0x7fe748108500 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.538+0000 7fe746000700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] conn(0x7fe72c0ade90 0x7fe72c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:36.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.539+0000 7fe746a00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe72c0a2cb0 0x7fe72c0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe738006010 tx=0x7fe73800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:36.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.539+0000 7fe746000700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] conn(0x7fe72c0ade90 0x7fe72c0b0330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:36.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.540+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== osd.0 v2:172.21.15.94:6802/837949984 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe74001fe50 con 0x7fe72c0ade90 2024-06-03T01:07:36.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.558+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 --> [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe748103bc0 con 0x7fe72c0ade90 2024-06-03T01:07:36.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74ca00700 1 -- 172.21.15.94:0/3921884257 <== osd.0 v2:172.21.15.94:6802/837949984 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe748103bc0 con 0x7fe72c0ade90 2024-06-03T01:07:36.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] conn(0x7fe72c0ade90 msgr2=0x7fe72c0b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] conn(0x7fe72c0ade90 0x7fe72c0b0330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe72c0a2cb0 msgr2=0x7fe72c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe72c0a2cb0 0x7fe72c0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe738006010 tx=0x7fe73800b260 comp rx=0 tx=0).stop 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 msgr2=0x7fe74819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 --2- 172.21.15.94:0/3921884257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe748108500 0x7fe74819f730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe74000ef30 tx=0x7fe74000c3a0 comp rx=0 tx=0).stop 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe747400700 1 -- 172.21.15.94:0/3921884257 reap_dead start 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 shutdown_connections 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 >> 172.21.15.94:0/3921884257 conn(0x7fe748078340 msgr2=0x7fe74810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.559+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 shutdown_connections 2024-06-03T01:07:36.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:34.560+0000 7fe74de00700 1 -- 172.21.15.94:0/3921884257 wait complete. 2024-06-03T01:07:36.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:36 smithi094 ceph-mon[32119]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-06-03T01:07:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:36 smithi107 ceph-mon[36622]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-06-03T01:07:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:36 smithi179 ceph-mon[37041]: pgmap v421: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-06-03T01:07:38.298 INFO:teuthology.orchestra.run.smithi094.stdout:223338299464 2024-06-03T01:07:38.299 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-03T01:07:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:38 smithi107 ceph-mon[36622]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T01:07:38.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:38 smithi094 ceph-mon[32119]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T01:07:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:38 smithi179 ceph-mon[37041]: pgmap v422: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-06-03T01:07:38.960 INFO:teuthology.orchestra.run.smithi094.stdout:167503724628 2024-06-03T01:07:38.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-03T01:07:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 -- 172.21.15.94:0/1667202308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 msgr2=0x7ff590107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 --2- 172.21.15.94:0/1667202308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 0x7ff590107020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0099c0 tx=0x7ff57c009cd0 comp rx=0 tx=0).stop 2024-06-03T01:07:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 -- 172.21.15.94:0/1667202308 shutdown_connections 2024-06-03T01:07:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 --2- 172.21.15.94:0/1667202308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff590102620 0x7ff590102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 --2- 172.21.15.94:0/1667202308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff590107700 0x7ff59010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:39.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 --2- 172.21.15.94:0/1667202308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 0x7ff590107020 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 -- 172.21.15.94:0/1667202308 >> 172.21.15.94:0/1667202308 conn(0x7ff5900fdf10 msgr2=0x7ff590100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 -- 172.21.15.94:0/1667202308 shutdown_connections 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.196+0000 7ff58fe00700 1 -- 172.21.15.94:0/1667202308 wait complete. 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.197+0000 7ff58fe00700 1 Processor -- start 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.197+0000 7ff58fe00700 1 -- start start 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff590102620 0x7ff59019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58fe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 0x7ff59019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff590107700 0x7ff59019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58fe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff590112730 con 0x7ff590107700 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901125b0 con 0x7ff590102620 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58fe00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901128b0 con 0x7ff590102fd0 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff590102620 0x7ff59019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:39.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58e000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 0x7ff59019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff590107700 0x7ff59019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58e000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 0x7ff59019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:37628/0 (socket says 172.21.15.94:37628) 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff590102620 0x7ff59019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:35168/0 (socket says 172.21.15.94:35168) 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.198+0000 7ff58e000700 1 -- 172.21.15.94:0/3938113618 learned_addr learned my addr 172.21.15.94:0/3938113618 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58e000700 1 -- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff590102620 msgr2=0x7ff59019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58e000700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff590102620 0x7ff59019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58e000700 1 -- 172.21.15.94:0/3938113618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff590107700 msgr2=0x7ff59019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58e000700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff590107700 0x7ff59019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58e000700 1 -- 172.21.15.94:0/3938113618 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59019fe40 con 0x7ff590102fd0 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58ea00700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff590102620 0x7ff59019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58e000700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 0x7ff59019b370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005f50 tx=0x7ff57c003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c003680 con 0x7ff590102fd0 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.199+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5901a00d0 con 0x7ff590102fd0 2024-06-03T01:07:39.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.200+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff590109280 con 0x7ff590102fd0 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.200+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c00b790 con 0x7ff590102fd0 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.200+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00f8a0 con 0x7ff590102fd0 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.201+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff570001030 con 0x7ff590102fd0 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.204+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff57c00fa40 con 0x7ff590102fd0 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.206+0000 7ff587400700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff5740a2e50 0x7ff5740a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.206+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff57c0d0370 con 0x7ff590102fd0 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.206+0000 7ff58ea00700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff5740a2e50 0x7ff5740a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.206+0000 7ff587400700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] conn(0x7ff5740ae030 0x7ff5740b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:39.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.206+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 --> [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff57c00b0b0 con 0x7ff5740ae030 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.206+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff57c0d0850 con 0x7ff590102fd0 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.207+0000 7ff58ea00700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff5740a2e50 0x7ff5740a5310 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff580009cb0 tx=0x7ff580009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.207+0000 7ff58f400700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] conn(0x7ff5740ae030 0x7ff5740b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.207+0000 7ff58f400700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] conn(0x7ff5740ae030 0x7ff5740b04d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.208+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== osd.1 v2:172.21.15.107:6800/3097634475 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff57c00b0b0 con 0x7ff5740ae030 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.226+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 --> [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff570002e60 con 0x7ff5740ae030 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff587400700 1 -- 172.21.15.94:0/3938113618 <== osd.1 v2:172.21.15.107:6800/3097634475 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff570002e60 con 0x7ff5740ae030 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] conn(0x7ff5740ae030 msgr2=0x7ff5740b04d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff58fe00700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] conn(0x7ff5740ae030 0x7ff5740b04d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff5740a2e50 msgr2=0x7ff5740a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff58fe00700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff5740a2e50 0x7ff5740a5310 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff580009cb0 tx=0x7ff580009260 comp rx=0 tx=0).stop 2024-06-03T01:07:39.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 msgr2=0x7ff59019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff58fe00700 1 --2- 172.21.15.94:0/3938113618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff590102fd0 0x7ff59019b370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005f50 tx=0x7ff57c003a90 comp rx=0 tx=0).stop 2024-06-03T01:07:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.227+0000 7ff58f400700 1 -- 172.21.15.94:0/3938113618 reap_dead start 2024-06-03T01:07:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.228+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 shutdown_connections 2024-06-03T01:07:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.228+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 >> 172.21.15.94:0/3938113618 conn(0x7ff5900fdf10 msgr2=0x7ff59010dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.228+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 shutdown_connections 2024-06-03T01:07:39.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:33.228+0000 7ff58fe00700 1 -- 172.21.15.94:0/3938113618 wait complete. 2024-06-03T01:07:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:40 smithi094 ceph-mon[32119]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:40.737 INFO:teuthology.orchestra.run.smithi094.stdout:55834574954 2024-06-03T01:07:40.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-03T01:07:40.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:40 smithi107 ceph-mon[36622]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:40.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 -- 172.21.15.94:0/3953695735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 msgr2=0x7ff260102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:40.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 --2- 172.21.15.94:0/3953695735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 0x7ff260102b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff2500099c0 tx=0x7ff250009cd0 comp rx=0 tx=0).stop 2024-06-03T01:07:40.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 -- 172.21.15.94:0/3953695735 shutdown_connections 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 --2- 172.21.15.94:0/3953695735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 0x7ff260102b20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 --2- 172.21.15.94:0/3953695735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff260107840 0x7ff26010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 --2- 172.21.15.94:0/3953695735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2601030f0 0x7ff260107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 -- 172.21.15.94:0/3953695735 >> 172.21.15.94:0/3953695735 conn(0x7ff2600795f0 msgr2=0x7ff260079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 -- 172.21.15.94:0/3953695735 shutdown_connections 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.842+0000 7ff266800700 1 -- 172.21.15.94:0/3953695735 wait complete. 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.843+0000 7ff266800700 1 Processor -- start 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.843+0000 7ff266800700 1 -- start start 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff266800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 0x7ff260074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff266800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2601030f0 0x7ff260074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff266800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff260107840 0x7ff260075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:40.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff266800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff260112760 con 0x7ff260107840 2024-06-03T01:07:40.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff266800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2601125e0 con 0x7ff260102740 2024-06-03T01:07:40.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff266800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff2601128e0 con 0x7ff2601030f0 2024-06-03T01:07:40.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 0x7ff260074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:40.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 0x7ff260074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57146/0 (socket says 172.21.15.94:57146) 2024-06-03T01:07:40.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25f400700 1 -- 172.21.15.94:0/2993903953 learned_addr learned my addr 172.21.15.94:0/2993903953 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25ea00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2601030f0 0x7ff260074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25fe00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff260107840 0x7ff260075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25f400700 1 -- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2601030f0 msgr2=0x7ff260074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25f400700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2601030f0 0x7ff260074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25f400700 1 -- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff260107840 msgr2=0x7ff260075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.844+0000 7ff25f400700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff260107840 0x7ff260075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.845+0000 7ff25f400700 1 -- 172.21.15.94:0/2993903953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2601a70f0 con 0x7ff260102740 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.845+0000 7ff25f400700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 0x7ff260074790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff250006010 tx=0x7ff25000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.845+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250003680 con 0x7ff260102740 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.845+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2500055b0 con 0x7ff260102740 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.845+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250003cc0 con 0x7ff260102740 2024-06-03T01:07:40.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.845+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2601a7380 con 0x7ff260102740 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.845+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2601a7830 con 0x7ff260102740 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.846+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff260062c20 con 0x7ff260102740 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.848+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff25000fb80 con 0x7ff260102740 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.850+0000 7ff257e00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff2440a2ed0 0x7ff2440a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.850+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ff2500d0600 con 0x7ff260102740 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.850+0000 7ff25ea00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff2440a2ed0 0x7ff2440a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.850+0000 7ff257e00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] conn(0x7ff2440ae0b0 0x7ff2440b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.850+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 --> [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff25000b790 con 0x7ff2440ae0b0 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.850+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7ff2500d09b0 con 0x7ff260102740 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.850+0000 7ff25ea00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff2440a2ed0 0x7ff2440a5390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff260101fc0 tx=0x7ff24c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.851+0000 7ff25fe00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] conn(0x7ff2440ae0b0 0x7ff2440b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.851+0000 7ff25fe00700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] conn(0x7ff2440ae0b0 0x7ff2440b0550 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:40.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.852+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== osd.2 v2:172.21.15.179:6800/3335947212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff25000b790 con 0x7ff2440ae0b0 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.870+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 --> [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2600630a0 con 0x7ff2440ae0b0 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.871+0000 7ff257e00700 1 -- 172.21.15.94:0/2993903953 <== osd.2 v2:172.21.15.179:6800/3335947212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff2600630a0 con 0x7ff2440ae0b0 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.871+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] conn(0x7ff2440ae0b0 msgr2=0x7ff2440b0550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.871+0000 7ff266800700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] conn(0x7ff2440ae0b0 0x7ff2440b0550 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff2440a2ed0 msgr2=0x7ff2440a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ff2440a2ed0 0x7ff2440a5390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff260101fc0 tx=0x7ff24c009260 comp rx=0 tx=0).stop 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 msgr2=0x7ff260074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 --2- 172.21.15.94:0/2993903953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff260102740 0x7ff260074790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff250006010 tx=0x7ff25000f690 comp rx=0 tx=0).stop 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff25fe00700 1 -- 172.21.15.94:0/2993903953 reap_dead start 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 shutdown_connections 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 >> 172.21.15.94:0/2993903953 conn(0x7ff2600795f0 msgr2=0x7ff260108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 shutdown_connections 2024-06-03T01:07:40.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:39.872+0000 7ff266800700 1 -- 172.21.15.94:0/2993903953 wait complete. 2024-06-03T01:07:40.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:40 smithi179 ceph-mon[37041]: pgmap v423: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:41.350 INFO:teuthology.orchestra.run.smithi094.stdout:34359738479 2024-06-03T01:07:41.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-03T01:07:42.600 INFO:teuthology.orchestra.run.smithi094.stdout:141733920857 2024-06-03T01:07:42.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-03T01:07:42.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:42 smithi094 ceph-mon[32119]: pgmap v424: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:42 smithi107 ceph-mon[36622]: pgmap v424: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:42.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:42 smithi179 ceph-mon[37041]: pgmap v424: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:43.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 -- 172.21.15.94:0/1915171502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 msgr2=0x7f5eb406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 --2- 172.21.15.94:0/1915171502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 0x7f5eb406a5d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0099c0 tx=0x7f5e9c009cd0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 -- 172.21.15.94:0/1915171502 shutdown_connections 2024-06-03T01:07:43.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 --2- 172.21.15.94:0/1915171502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb41084e0 0x7f5eb41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 --2- 172.21.15.94:0/1915171502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb406ab10 0x7f5eb410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 --2- 172.21.15.94:0/1915171502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 0x7f5eb406a5d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 -- 172.21.15.94:0/1915171502 >> 172.21.15.94:0/1915171502 conn(0x7f5eb40782c0 msgr2=0x7f5eb40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 -- 172.21.15.94:0/1915171502 shutdown_connections 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.411+0000 7f5eba400700 1 -- 172.21.15.94:0/1915171502 wait complete. 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.412+0000 7f5eba400700 1 Processor -- start 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eba400700 1 -- start start 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eba400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 0x7f5eb419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eba400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb406ab10 0x7f5eb419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eba400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb41084e0 0x7f5eb419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eba400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb41127b0 con 0x7f5eb406a170 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eba400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4112630 con 0x7f5eb406ab10 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eba400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb4112930 con 0x7f5eb41084e0 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eb3400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 0x7f5eb419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eb3400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 0x7f5eb419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60000/0 (socket says 172.21.15.94:60000) 2024-06-03T01:07:43.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eb3400700 1 -- 172.21.15.94:0/3934863069 learned_addr learned my addr 172.21.15.94:0/3934863069 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eaaa00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb406ab10 0x7f5eb419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.413+0000 7f5eb3e00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb41084e0 0x7f5eb419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb3400700 1 -- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb41084e0 msgr2=0x7f5eb419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb3400700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5eb41084e0 0x7f5eb419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb3400700 1 -- 172.21.15.94:0/3934863069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb406ab10 msgr2=0x7f5eb419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb3400700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb406ab10 0x7f5eb419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb3400700 1 -- 172.21.15.94:0/3934863069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb419fe50 con 0x7f5eb406a170 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb3400700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 0x7f5eb419ae40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e9a0 tx=0x7f5ea400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400ca10 con 0x7f5eb406a170 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea400cbb0 con 0x7f5eb406a170 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.414+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4018680 con 0x7f5eb406a170 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.415+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb41a0140 con 0x7f5eb406a170 2024-06-03T01:07:43.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.415+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb41a79c0 con 0x7f5eb406a170 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.415+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5eb4062c20 con 0x7f5eb406a170 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.418+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5ea4010430 con 0x7f5eb406a170 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.419+0000 7f5eb0c00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5e980a2cb0 0x7f5e980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.419+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f5ea40cff60 con 0x7f5eb406a170 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.419+0000 7f5eaaa00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5e980a2cb0 0x7f5e980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.419+0000 7f5eb0c00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] conn(0x7f5e980ade90 0x7f5e980b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.419+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 --> [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ea401bec0 con 0x7f5e980ade90 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.420+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f5ea40d0310 con 0x7f5eb406a170 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.420+0000 7f5eb3e00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] conn(0x7f5e980ade90 0x7f5e980b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.420+0000 7f5eaaa00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5e980a2cb0 0x7f5e980a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40788c0 tx=0x7f5e9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.421+0000 7f5eb3e00700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] conn(0x7f5e980ade90 0x7f5e980b0330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:43.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.422+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== osd.8 v2:172.21.15.179:6816/3410192696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ea401bec0 con 0x7f5e980ade90 2024-06-03T01:07:43.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.440+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 --> [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5eb40630a0 con 0x7f5e980ade90 2024-06-03T01:07:43.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.440+0000 7f5eb0c00700 1 -- 172.21.15.94:0/3934863069 <== osd.8 v2:172.21.15.179:6816/3410192696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5eb40630a0 con 0x7f5e980ade90 2024-06-03T01:07:43.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] conn(0x7f5e980ade90 msgr2=0x7f5e980b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] conn(0x7f5e980ade90 0x7f5e980b0330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5e980a2cb0 msgr2=0x7f5e980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f5e980a2cb0 0x7f5e980a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40788c0 tx=0x7f5e9c005fb0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 msgr2=0x7f5eb419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 --2- 172.21.15.94:0/3934863069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5eb406a170 0x7f5eb419ae40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e9a0 tx=0x7f5ea400ecb0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eb3e00700 1 -- 172.21.15.94:0/3934863069 reap_dead start 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 shutdown_connections 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.441+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 >> 172.21.15.94:0/3934863069 conn(0x7f5eb40782c0 msgr2=0x7f5eb410d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.442+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 shutdown_connections 2024-06-03T01:07:43.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.442+0000 7f5eba400700 1 -- 172.21.15.94:0/3934863069 wait complete. 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 -- 172.21.15.94:0/2641723335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 msgr2=0x7fa4901088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 --2- 172.21.15.94:0/2641723335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 0x7fa4901088e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa480009990 tx=0x7fa480009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 -- 172.21.15.94:0/2641723335 shutdown_connections 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 --2- 172.21.15.94:0/2641723335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa49006ab30 0x7fa49010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 --2- 172.21.15.94:0/2641723335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 0x7fa4901088e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 --2- 172.21.15.94:0/2641723335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49006a170 0x7fa49006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 -- 172.21.15.94:0/2641723335 >> 172.21.15.94:0/2641723335 conn(0x7fa490078340 msgr2=0x7fa490078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 -- 172.21.15.94:0/2641723335 shutdown_connections 2024-06-03T01:07:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.893+0000 7fa497c00700 1 -- 172.21.15.94:0/2641723335 wait complete. 2024-06-03T01:07:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.894+0000 7fa497c00700 1 Processor -- start 2024-06-03T01:07:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa497c00700 1 -- start start 2024-06-03T01:07:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa497c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa49006a170 0x7fa49019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa497c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49006ab30 0x7fa49019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa497c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 0x7fa490198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa497c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901127b0 con 0x7fa490108500 2024-06-03T01:07:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa497c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa490112630 con 0x7fa49006a170 2024-06-03T01:07:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa497c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa490112930 con 0x7fa49006ab30 2024-06-03T01:07:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa495200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa49006a170 0x7fa49019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa495c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 0x7fa490198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa49006a170 0x7fa49019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57170/0 (socket says 172.21.15.94:57170) 2024-06-03T01:07:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.895+0000 7fa487e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49006ab30 0x7fa49019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 -- 172.21.15.94:0/718919195 learned_addr learned my addr 172.21.15.94:0/718919195 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 0x7fa490198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60006/0 (socket says 172.21.15.94:60006) 2024-06-03T01:07:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 -- 172.21.15.94:0/718919195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49006ab30 msgr2=0x7fa49019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa49006ab30 0x7fa49019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 -- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 msgr2=0x7fa490198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 0x7fa490198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 -- 172.21.15.94:0/718919195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4901991f0 con 0x7fa49006a170 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495c00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa490108500 0x7fa490198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.896+0000 7fa495200700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa49006a170 0x7fa49019e880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa48000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.897+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480027070 con 0x7fa49006a170 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.897+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4800056d0 con 0x7fa49006a170 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.897+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa490199480 con 0x7fa49006a170 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.897+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480022e10 con 0x7fa49006a170 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.897+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa490101390 con 0x7fa49006a170 2024-06-03T01:07:43.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.898+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa490062c20 con 0x7fa49006a170 2024-06-03T01:07:43.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.900+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa48002a070 con 0x7fa49006a170 2024-06-03T01:07:43.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.901+0000 7fa48e000700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa4740a2cb0 0x7fa4740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.902+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa4800d0ae0 con 0x7fa49006a170 2024-06-03T01:07:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.902+0000 7fa487e00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa4740a2cb0 0x7fa4740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.902+0000 7fa48e000700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] conn(0x7fa4740ade90 0x7fa4740b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.902+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 --> [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4740b0a00 con 0x7fa4740ade90 2024-06-03T01:07:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.902+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa4800d0e90 con 0x7fa49006a170 2024-06-03T01:07:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.902+0000 7fa495c00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] conn(0x7fa4740ade90 0x7fa4740b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:43.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.902+0000 7fa487e00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa4740a2cb0 0x7fa4740a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa478009680 tx=0x7fa478012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.903+0000 7fa495c00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] conn(0x7fa4740ade90 0x7fa4740b0330 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:43.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.904+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== osd.9 v2:172.21.15.94:6826/165849117 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4740b0a00 con 0x7fa4740ade90 2024-06-03T01:07:43.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.921+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 --> [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4900630a0 con 0x7fa4740ade90 2024-06-03T01:07:43.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.922+0000 7fa48e000700 1 -- 172.21.15.94:0/718919195 <== osd.9 v2:172.21.15.94:6826/165849117 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa4900630a0 con 0x7fa4740ade90 2024-06-03T01:07:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.922+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] conn(0x7fa4740ade90 msgr2=0x7fa4740b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.922+0000 7fa497c00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] conn(0x7fa4740ade90 0x7fa4740b0330 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa4740a2cb0 msgr2=0x7fa4740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa4740a2cb0 0x7fa4740a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa478009680 tx=0x7fa478012040 comp rx=0 tx=0).stop 2024-06-03T01:07:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa49006a170 msgr2=0x7fa49019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 --2- 172.21.15.94:0/718919195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa49006a170 0x7fa49019e880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa480000c00 tx=0x7fa48000bab0 comp rx=0 tx=0).stop 2024-06-03T01:07:43.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa495c00700 1 -- 172.21.15.94:0/718919195 reap_dead start 2024-06-03T01:07:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 shutdown_connections 2024-06-03T01:07:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 >> 172.21.15.94:0/718919195 conn(0x7fa490078340 msgr2=0x7fa4900ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 shutdown_connections 2024-06-03T01:07:43.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:43.923+0000 7fa497c00700 1 -- 172.21.15.94:0/718919195 wait complete. 2024-06-03T01:07:44.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:44 smithi094 ceph-mon[32119]: pgmap v425: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T01:07:44.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:44 smithi107 ceph-mon[36622]: pgmap v425: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T01:07:44.799 INFO:teuthology.orchestra.run.smithi094.stdout:77309411430 2024-06-03T01:07:44.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 -- 172.21.15.94:0/3502383129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3518074830 msgr2=0x7f3518074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 --2- 172.21.15.94:0/3502383129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3518074830 0x7f3518074c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3508009990 tx=0x7f3508009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 -- 172.21.15.94:0/3502383129 shutdown_connections 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 --2- 172.21.15.94:0/3502383129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35181133d0 0x7f35181157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 --2- 172.21.15.94:0/3502383129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3518074830 0x7f3518074c10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 --2- 172.21.15.94:0/3502383129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3518075150 0x7f3518112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 -- 172.21.15.94:0/3502383129 >> 172.21.15.94:0/3502383129 conn(0x7f35180fdf10 msgr2=0x7f3518100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 -- 172.21.15.94:0/3502383129 shutdown_connections 2024-06-03T01:07:44.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.139+0000 7f3517e00700 1 -- 172.21.15.94:0/3502383129 wait complete. 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.140+0000 7f3517e00700 1 Processor -- start 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517e00700 1 -- start start 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3518074830 0x7f351819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3518075150 0x7f351819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35181133d0 0x7f35181a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3518118d00 con 0x7f35181133d0 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3518118b80 con 0x7f3518074830 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3518118e80 con 0x7f3518075150 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35181133d0 0x7f35181a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35181133d0 0x7f35181a3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60024/0 (socket says 172.21.15.94:60024) 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3517400700 1 -- 172.21.15.94:0/2897505987 learned_addr learned my addr 172.21.15.94:0/2897505987 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:44.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f3516a00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3518074830 0x7f351819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.141+0000 7f350fe00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3518075150 0x7f351819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f3517400700 1 -- 172.21.15.94:0/2897505987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3518075150 msgr2=0x7f351819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f3517400700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3518075150 0x7f351819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f3517400700 1 -- 172.21.15.94:0/2897505987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3518074830 msgr2=0x7f351819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f3517400700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3518074830 0x7f351819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f3517400700 1 -- 172.21.15.94:0/2897505987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35181a41c0 con 0x7f35181133d0 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f3517400700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35181133d0 0x7f35181a3a80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f351000c110 tx=0x7f351000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35100092e0 con 0x7f35181133d0 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3510003a00 con 0x7f35181133d0 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35181a4450 con 0x7f35181133d0 2024-06-03T01:07:44.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.142+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510004680 con 0x7f35181133d0 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.143+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35181abd30 con 0x7f35181133d0 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.143+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3518062be0 con 0x7f35181133d0 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.145+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f351001a020 con 0x7f35181133d0 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.147+0000 7f350f400700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34fc0a2d30 0x7f34fc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.147+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f35100cf800 con 0x7f35181133d0 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.147+0000 7f3516a00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34fc0a2d30 0x7f34fc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.148+0000 7f350f400700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] conn(0x7f34fc0adf10 0x7f34fc0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:44.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.148+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 --> [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3510009c20 con 0x7f34fc0adf10 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.148+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f35100cfbb0 con 0x7f35181133d0 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.148+0000 7f350fe00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] conn(0x7f34fc0adf10 0x7f34fc0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.148+0000 7f3516a00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34fc0a2d30 0x7f34fc0a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f35080098e0 tx=0x7f3508005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.148+0000 7f350fe00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] conn(0x7f34fc0adf10 0x7f34fc0b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.149+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== osd.6 v2:172.21.15.94:6818/3118996469 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3510009c20 con 0x7f34fc0adf10 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.167+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 --> [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3518063060 con 0x7f34fc0adf10 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.168+0000 7f350f400700 1 -- 172.21.15.94:0/2897505987 <== osd.6 v2:172.21.15.94:6818/3118996469 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3518063060 con 0x7f34fc0adf10 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.168+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] conn(0x7f34fc0adf10 msgr2=0x7f34fc0b03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.168+0000 7f3517e00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] conn(0x7f34fc0adf10 0x7f34fc0b03b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34fc0a2d30 msgr2=0x7f34fc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34fc0a2d30 0x7f34fc0a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f35080098e0 tx=0x7f3508005960 comp rx=0 tx=0).stop 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35181133d0 msgr2=0x7f35181a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 --2- 172.21.15.94:0/2897505987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35181133d0 0x7f35181a3a80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f351000c110 tx=0x7f351000c420 comp rx=0 tx=0).stop 2024-06-03T01:07:44.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517400700 1 -- 172.21.15.94:0/2897505987 reap_dead start 2024-06-03T01:07:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 shutdown_connections 2024-06-03T01:07:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 >> 172.21.15.94:0/2897505987 conn(0x7f35180fdf10 msgr2=0x7f35180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 shutdown_connections 2024-06-03T01:07:44.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:44.169+0000 7f3517e00700 1 -- 172.21.15.94:0/2897505987 wait complete. 2024-06-03T01:07:44.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:44 smithi179 ceph-mon[37041]: pgmap v425: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-06-03T01:07:45.997 INFO:teuthology.orchestra.run.smithi094.stdout:292057776191 2024-06-03T01:07:45.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-03T01:07:46.331 INFO:teuthology.orchestra.run.smithi094.stdout:257698037829 2024-06-03T01:07:46.332 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-03T01:07:46.551 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:46 smithi094 ceph-mon[32119]: pgmap v426: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-03T01:07:46.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 -- 172.21.15.94:0/1750951679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 msgr2=0x7f803810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:46.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 --2- 172.21.15.94:0/1750951679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 0x7f803810ea00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8024009990 tx=0x7f8024009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 -- 172.21.15.94:0/1750951679 shutdown_connections 2024-06-03T01:07:46.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 --2- 172.21.15.94:0/1750951679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f803806a1a0 0x7f803806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 --2- 172.21.15.94:0/1750951679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803810ef40 0x7f8038111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 --2- 172.21.15.94:0/1750951679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 0x7f803810ea00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 -- 172.21.15.94:0/1750951679 >> 172.21.15.94:0/1750951679 conn(0x7f8038078340 msgr2=0x7f8038078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 -- 172.21.15.94:0/1750951679 shutdown_connections 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.614+0000 7f803f800700 1 -- 172.21.15.94:0/1750951679 wait complete. 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.615+0000 7f803f800700 1 Processor -- start 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803f800700 1 -- start start 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803f800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f803806a1a0 0x7f803810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803f800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 0x7f8038105990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803f800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803810ef40 0x7f8038105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803f800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f80381051d0 con 0x7f803810ef40 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803f800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8038105050 con 0x7f803806a1a0 2024-06-03T01:07:46.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803f800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8038105350 con 0x7f803806aac0 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803d800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803810ef40 0x7f8038105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803ce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f803806a1a0 0x7f803810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803ce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f803806a1a0 0x7f803810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:35406/0 (socket says 172.21.15.94:35406) 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f803ce00700 1 -- 172.21.15.94:0/53896389 learned_addr learned my addr 172.21.15.94:0/53896389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.616+0000 7f8037e00700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 0x7f8038105990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f803d800700 1 -- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 msgr2=0x7f8038105990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f803d800700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 0x7f8038105990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f803d800700 1 -- 172.21.15.94:0/53896389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f803806a1a0 msgr2=0x7f803810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f803d800700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f803806a1a0 0x7f803810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f803d800700 1 -- 172.21.15.94:0/53896389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8038106490 con 0x7f803810ef40 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f803ce00700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f803806a1a0 0x7f803810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:46.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f8037e00700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f803806aac0 0x7f8038105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f803d800700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803810ef40 0x7f8038105ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8038069d60 tx=0x7f803000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.617+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030003a00 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.618+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8038106780 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.618+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8030003ba0 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.618+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803810a330 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.618+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80300047b0 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.619+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f803810a960 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.621+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8030012050 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.622+0000 7f8035600700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f80200a2cb0 0x7f80200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.622+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f80300cf2a0 con 0x7f803810ef40 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.622+0000 7f803ce00700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f80200a2cb0 0x7f80200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.623+0000 7f8035600700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] conn(0x7f80200ade90 0x7f80200b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:46.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.623+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 --> [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8030011e50 con 0x7f80200ade90 2024-06-03T01:07:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.623+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f80300cf650 con 0x7f803810ef40 2024-06-03T01:07:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.623+0000 7f8037e00700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] conn(0x7f80200ade90 0x7f80200b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.623+0000 7f803ce00700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f80200a2cb0 0x7f80200a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f802c011550 tx=0x7f802c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.624+0000 7f8037e00700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] conn(0x7f80200ade90 0x7f80200b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.625+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== osd.11 v2:172.21.15.179:6824/4289321509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8030011e50 con 0x7f80200ade90 2024-06-03T01:07:46.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.642+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 --> [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f803810ade0 con 0x7f80200ade90 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f8035600700 1 -- 172.21.15.94:0/53896389 <== osd.11 v2:172.21.15.179:6824/4289321509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f803810ade0 con 0x7f80200ade90 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] conn(0x7f80200ade90 msgr2=0x7f80200b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f803f800700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] conn(0x7f80200ade90 0x7f80200b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f80200a2cb0 msgr2=0x7f80200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f803f800700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f80200a2cb0 0x7f80200a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f802c011550 tx=0x7f802c009260 comp rx=0 tx=0).stop 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803810ef40 msgr2=0x7f8038105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f803f800700 1 --2- 172.21.15.94:0/53896389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803810ef40 0x7f8038105ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8038069d60 tx=0x7f803000c3a0 comp rx=0 tx=0).stop 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.643+0000 7f803d800700 1 -- 172.21.15.94:0/53896389 reap_dead start 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.644+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 shutdown_connections 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.644+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 >> 172.21.15.94:0/53896389 conn(0x7f8038078340 msgr2=0x7f80381103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.644+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 shutdown_connections 2024-06-03T01:07:46.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:46.644+0000 7f803f800700 1 -- 172.21.15.94:0/53896389 wait complete. 2024-06-03T01:07:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:46 smithi107 ceph-mon[36622]: pgmap v426: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-03T01:07:46.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:46 smithi179 ceph-mon[37041]: pgmap v426: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-06-03T01:07:47.534 INFO:teuthology.orchestra.run.smithi094.stdout:313532612663 2024-06-03T01:07:47.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-03T01:07:48.074 INFO:teuthology.orchestra.run.smithi094.stdout:201863462992 2024-06-03T01:07:48.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-03T01:07:48.677 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:48.682 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:48.728 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:48.736 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:48 smithi094 ceph-mon[32119]: pgmap v427: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-03T01:07:48.741 INFO:teuthology.orchestra.run.smithi094.stdout:347892351028 2024-06-03T01:07:48.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-03T01:07:48.755 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:48.755 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:48 smithi107 ceph-mon[36622]: pgmap v427: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-03T01:07:48.787 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:48 smithi179 ceph-mon[37041]: pgmap v427: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-03T01:07:49.267 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:49.271 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:49.274 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:07:50.468 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:50 smithi094 ceph-mon[32119]: pgmap v428: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:50.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:50 smithi107 ceph-mon[36622]: pgmap v428: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:50 smithi179 ceph-mon[37041]: pgmap v428: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-06-03T01:07:52.702 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:52 smithi094 ceph-mon[32119]: pgmap v429: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:52 smithi107 ceph-mon[36622]: pgmap v429: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:52 smithi179 ceph-mon[37041]: pgmap v429: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:07:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:54 smithi107 ceph-mon[36622]: pgmap v430: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:07:54.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:54 smithi094 ceph-mon[32119]: pgmap v430: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:07:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:54 smithi179 ceph-mon[37041]: pgmap v430: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:07:56.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:55 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:07:56.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:55 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:07:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:55 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:07:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: pgmap v431: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T01:07:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:07:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:07:57.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: pgmap v431: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T01:07:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:07:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:07:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: pgmap v431: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:07:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:07:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 -- 172.21.15.94:0/333889576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 msgr2=0x7fbea0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 --2- 172.21.15.94:0/333889576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 0x7fbea0074aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009990 tx=0x7fbe90009ca0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 -- 172.21.15.94:0/333889576 shutdown_connections 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 --2- 172.21.15.94:0/333889576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea010ee90 0x7fbea0111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 --2- 172.21.15.94:0/333889576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 0x7fbea0074aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 --2- 172.21.15.94:0/333889576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea0074fe0 0x7fbea010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 -- 172.21.15.94:0/333889576 >> 172.21.15.94:0/333889576 conn(0x7fbea0079640 msgr2=0x7fbea0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.057+0000 7fbea6e00700 1 -- 172.21.15.94:0/333889576 shutdown_connections 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.058+0000 7fbea6e00700 1 -- 172.21.15.94:0/333889576 wait complete. 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.058+0000 7fbea6e00700 1 Processor -- start 2024-06-03T01:07:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbea6e00700 1 -- start start 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbea6e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 0x7fbea019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbea6e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea0074fe0 0x7fbea019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbea6e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea010ee90 0x7fbea019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbea6e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01149e0 con 0x7fbea00746c0 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbea6e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbea0114860 con 0x7fbea010ee90 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbea6e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbea0114b60 con 0x7fbea0074fe0 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.059+0000 7fbe9fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 0x7fbea019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbe97e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea0074fe0 0x7fbea019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea010ee90 0x7fbea019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea010ee90 0x7fbea019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:59178/0 (socket says 172.21.15.94:59178) 2024-06-03T01:07:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbe9fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 0x7fbea019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33250/0 (socket says 172.21.15.94:33250) 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbe97e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea0074fe0 0x7fbea019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:47354/0 (socket says 172.21.15.94:47354) 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 -- 172.21.15.94:0/2020150855 learned_addr learned my addr 172.21.15.94:0/2020150855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 -- 172.21.15.94:0/2020150855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea0074fe0 msgr2=0x7fbea019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea0074fe0 0x7fbea019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 -- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 msgr2=0x7fbea019aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 0x7fbea019aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 -- 172.21.15.94:0/2020150855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea019ff00 con 0x7fbea010ee90 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbe97e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea0074fe0 0x7fbea019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.060+0000 7fbea4e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea010ee90 0x7fbea019f7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800ef90 tx=0x7fbe9800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.061+0000 7fbe9d600700 1 -- 172.21.15.94:0/2020150855 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe980092e0 con 0x7fbea010ee90 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.061+0000 7fbe9d600700 1 -- 172.21.15.94:0/2020150855 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe98004bb0 con 0x7fbea010ee90 2024-06-03T01:07:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.061+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea01a0190 con 0x7fbea010ee90 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.061+0000 7fbe9d600700 1 -- 172.21.15.94:0/2020150855 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98004280 con 0x7fbea010ee90 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.061+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea01a7a70 con 0x7fbea010ee90 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.062+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea0069180 con 0x7fbea010ee90 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.064+0000 7fbe9d600700 1 -- 172.21.15.94:0/2020150855 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbe9801b030 con 0x7fbea010ee90 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.065+0000 7fbe9d600700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbe840a2cb0 0x7fbe840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.065+0000 7fbe9d600700 1 -- 172.21.15.94:0/2020150855 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbe980cef90 con 0x7fbea010ee90 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.066+0000 7fbe9fe00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbe840a2cb0 0x7fbe840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.066+0000 7fbe9fe00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbe840a2cb0 0x7fbe840a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe900098e0 tx=0x7fbe90005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:07:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.070+0000 7fbe9d600700 1 -- 172.21.15.94:0/2020150855 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbe98022020 con 0x7fbea010ee90 2024-06-03T01:07:59.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.303+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbea002dca0 con 0x7fbea010ee90 2024-06-03T01:07:59.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.304+0000 7fbe9d600700 1 -- 172.21.15.94:0/2020150855 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbe98010040 con 0x7fbea010ee90 2024-06-03T01:07:59.304 INFO:teuthology.orchestra.run.smithi094.stdout:201863462995 2024-06-03T01:07:59.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbe840a2cb0 msgr2=0x7fbe840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:59.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbe840a2cb0 0x7fbe840a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbe900098e0 tx=0x7fbe90005960 comp rx=0 tx=0).stop 2024-06-03T01:07:59.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea010ee90 msgr2=0x7fbea019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:07:59.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea010ee90 0x7fbea019f7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800ef90 tx=0x7fbe9800c3a0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 shutdown_connections 2024-06-03T01:07:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbea010ee90 0x7fbea019f7c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbe840a2cb0 0x7fbe840a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbea00746c0 0x7fbea019aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 --2- 172.21.15.94:0/2020150855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbea0074fe0 0x7fbea019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:07:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 >> 172.21.15.94:0/2020150855 conn(0x7fbea0079640 msgr2=0x7fbea00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:07:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 shutdown_connections 2024-06-03T01:07:59.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:07:59.307+0000 7fbea6e00700 1 -- 172.21.15.94:0/2020150855 wait complete. 2024-06-03T01:07:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:59 smithi094 ceph-mon[32119]: pgmap v432: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:59.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:07:59 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2020150855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T01:07:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:59 smithi179 ceph-mon[37041]: pgmap v432: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:07:59 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2020150855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T01:07:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:59 smithi107 ceph-mon[36622]: pgmap v432: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:07:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:07:59 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2020150855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-03T01:08:00.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 -- 172.21.15.94:0/2112141453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461410a690 msgr2=0x7f461410aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:00.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 --2- 172.21.15.94:0/2112141453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461410a690 0x7f461410aa70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4608009990 tx=0x7f4608009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 -- 172.21.15.94:0/2112141453 shutdown_connections 2024-06-03T01:08:00.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 --2- 172.21.15.94:0/2112141453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4614105050 0x7f4614111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 --2- 172.21.15.94:0/2112141453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f461410a690 0x7f461410aa70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 --2- 172.21.15.94:0/2112141453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4614104690 0x7f4614104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 -- 172.21.15.94:0/2112141453 >> 172.21.15.94:0/2112141453 conn(0x7f46141001a0 msgr2=0x7f46141025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:00.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 -- 172.21.15.94:0/2112141453 shutdown_connections 2024-06-03T01:08:00.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.331+0000 7f461be00700 1 -- 172.21.15.94:0/2112141453 wait complete. 2024-06-03T01:08:00.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.332+0000 7f461be00700 1 Processor -- start 2024-06-03T01:08:00.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.332+0000 7f461be00700 1 -- start start 2024-06-03T01:08:00.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f461be00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4614104690 0x7f461419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:00.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f461be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4614105050 0x7f461419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:00.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f461be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f461410a690 0x7f461419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f461be00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46141149e0 con 0x7f4614105050 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f461be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4614114860 con 0x7f461410a690 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f461be00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4614114b60 con 0x7f4614104690 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f4618a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4614105050 0x7f461419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f4619400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4614104690 0x7f461419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f4618a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4614105050 0x7f461419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33262/0 (socket says 172.21.15.94:33262) 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f4618a00700 1 -- 172.21.15.94:0/3970127471 learned_addr learned my addr 172.21.15.94:0/3970127471 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:00.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.333+0000 7f4619e00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f461410a690 0x7f461419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:00.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4619400700 1 -- 172.21.15.94:0/3970127471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f461410a690 msgr2=0x7f461419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:00.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4619400700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f461410a690 0x7f461419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4619400700 1 -- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4614105050 msgr2=0x7f461419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4619400700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4614105050 0x7f461419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4619400700 1 -- 172.21.15.94:0/3970127471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461419fe20 con 0x7f4614104690 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4618a00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4614105050 0x7f461419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4619400700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4614104690 0x7f461419ae10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4608006010 tx=0x7f460800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4612000700 1 -- 172.21.15.94:0/3970127471 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608027070 con 0x7f4614104690 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46141a00b0 con 0x7f4614104690 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4612000700 1 -- 172.21.15.94:0/3970127471 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46080056d0 con 0x7f4614104690 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f4612000700 1 -- 172.21.15.94:0/3970127471 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608022d90 con 0x7f4614104690 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.334+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46141a7990 con 0x7f4614104690 2024-06-03T01:08:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.335+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4614105dd0 con 0x7f4614104690 2024-06-03T01:08:00.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.337+0000 7f4612000700 1 -- 172.21.15.94:0/3970127471 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f460802a070 con 0x7f4614104690 2024-06-03T01:08:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.339+0000 7f4612000700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f45fc0a2cb0 0x7f45fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.339+0000 7f4612000700 1 -- 172.21.15.94:0/3970127471 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f46080d0cb0 con 0x7f4614104690 2024-06-03T01:08:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.339+0000 7f4618a00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f45fc0a2cb0 0x7f45fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.340+0000 7f4618a00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f45fc0a2cb0 0x7f45fc0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4614109c30 tx=0x7f4600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:00.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.341+0000 7f4612000700 1 -- 172.21.15.94:0/3970127471 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4608031020 con 0x7f4614104690 2024-06-03T01:08:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.573+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f461402dca0 con 0x7f4614104690 2024-06-03T01:08:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.574+0000 7f4612000700 1 -- 172.21.15.94:0/3970127471 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4608013070 con 0x7f4614104690 2024-06-03T01:08:00.575 INFO:teuthology.orchestra.run.smithi094.stdout:257698037832 2024-06-03T01:08:00.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f45fc0a2cb0 msgr2=0x7f45fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:00.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f45fc0a2cb0 0x7f45fc0a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4614109c30 tx=0x7f4600009260 comp rx=0 tx=0).stop 2024-06-03T01:08:00.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4614104690 msgr2=0x7f461419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:00.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4614104690 0x7f461419ae10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4608006010 tx=0x7f460800bab0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 shutdown_connections 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f461410a690 0x7f461419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f45fc0a2cb0 0x7f45fc0a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4614104690 0x7f461419ae10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 --2- 172.21.15.94:0/3970127471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4614105050 0x7f461419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 >> 172.21.15.94:0/3970127471 conn(0x7f46141001a0 msgr2=0x7f46141006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 shutdown_connections 2024-06-03T01:08:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:00.577+0000 7f461be00700 1 -- 172.21.15.94:0/3970127471 wait complete. 2024-06-03T01:08:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:01 smithi094 ceph-mon[32119]: pgmap v433: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:01.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:01 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3970127471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T01:08:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:01 smithi179 ceph-mon[37041]: pgmap v433: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:01.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:01 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3970127471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T01:08:01.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:01 smithi107 ceph-mon[36622]: pgmap v433: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:01.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:01 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3970127471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-03T01:08:02.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.960+0000 7f321a000700 1 -- 172.21.15.94:0/3809813093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214104690 msgr2=0x7f3214104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.960+0000 7f321a000700 1 --2- 172.21.15.94:0/3809813093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214104690 0x7f3214104b10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3200009990 tx=0x7f3200009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.960+0000 7f321a000700 1 -- 172.21.15.94:0/3809813093 shutdown_connections 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.960+0000 7f321a000700 1 --2- 172.21.15.94:0/3809813093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f321410a690 0x7f321410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.960+0000 7f321a000700 1 --2- 172.21.15.94:0/3809813093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3214105050 0x7f3214111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.960+0000 7f321a000700 1 --2- 172.21.15.94:0/3809813093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214104690 0x7f3214104b10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.961+0000 7f321a000700 1 -- 172.21.15.94:0/3809813093 >> 172.21.15.94:0/3809813093 conn(0x7f32141001a0 msgr2=0x7f32141025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.961+0000 7f321a000700 1 -- 172.21.15.94:0/3809813093 shutdown_connections 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.961+0000 7f321a000700 1 -- 172.21.15.94:0/3809813093 wait complete. 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 Processor -- start 2024-06-03T01:08:02.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 -- start start 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3214104690 0x7f321419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214105050 0x7f321419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f321410a690 0x7f321419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f32141149c0 con 0x7f3214105050 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3214114840 con 0x7f321410a690 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f321a000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3214114b40 con 0x7f3214104690 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f3212a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214105050 0x7f321419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f3212a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214105050 0x7f321419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33276/0 (socket says 172.21.15.94:33276) 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.962+0000 7f3212a00700 1 -- 172.21.15.94:0/226536246 learned_addr learned my addr 172.21.15.94:0/226536246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3213400700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3214104690 0x7f321419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3213e00700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f321410a690 0x7f321419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:02.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3212a00700 1 -- 172.21.15.94:0/226536246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3214104690 msgr2=0x7f321419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3212a00700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3214104690 0x7f321419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3212a00700 1 -- 172.21.15.94:0/226536246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f321410a690 msgr2=0x7f321419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3212a00700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f321410a690 0x7f321419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3212a00700 1 -- 172.21.15.94:0/226536246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321419fe30 con 0x7f3214105050 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3213e00700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f321410a690 0x7f321419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.963+0000 7f3212a00700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214105050 0x7f321419b3b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f320000bae0 tx=0x7f320000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.964+0000 7f320be00700 1 -- 172.21.15.94:0/226536246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200026070 con 0x7f3214105050 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.964+0000 7f320be00700 1 -- 172.21.15.94:0/226536246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3200003d70 con 0x7f3214105050 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.964+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32141a00c0 con 0x7f3214105050 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.964+0000 7f320be00700 1 -- 172.21.15.94:0/226536246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200021410 con 0x7f3214105050 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.964+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32141a79a0 con 0x7f3214105050 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.965+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3214069180 con 0x7f3214105050 2024-06-03T01:08:02.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.967+0000 7f320be00700 1 -- 172.21.15.94:0/226536246 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3200029070 con 0x7f3214105050 2024-06-03T01:08:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.968+0000 7f320be00700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f31f80a2cb0 0x7f31f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.968+0000 7f320be00700 1 -- 172.21.15.94:0/226536246 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f32000d02c0 con 0x7f3214105050 2024-06-03T01:08:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.969+0000 7f3213400700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f31f80a2cb0 0x7f31f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.969+0000 7f3213400700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f31f80a2cb0 0x7f31f80a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f320400f540 tx=0x7f320400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:02.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:01.973+0000 7f320be00700 1 -- 172.21.15.94:0/226536246 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f320005d850 con 0x7f3214105050 2024-06-03T01:08:02.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.206+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f321402dca0 con 0x7f3214105050 2024-06-03T01:08:02.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.207+0000 7f320be00700 1 -- 172.21.15.94:0/226536246 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3200013070 con 0x7f3214105050 2024-06-03T01:08:02.207 INFO:teuthology.orchestra.run.smithi094.stdout:223338299470 2024-06-03T01:08:02.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f31f80a2cb0 msgr2=0x7f31f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:02.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f31f80a2cb0 0x7f31f80a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f320400f540 tx=0x7f320400b040 comp rx=0 tx=0).stop 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214105050 msgr2=0x7f321419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214105050 0x7f321419b3b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f320000bae0 tx=0x7f320000bb10 comp rx=0 tx=0).stop 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 shutdown_connections 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f321410a690 0x7f321419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f31f80a2cb0 0x7f31f80a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3214104690 0x7f321419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 --2- 172.21.15.94:0/226536246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3214105050 0x7f321419b3b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.210+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 >> 172.21.15.94:0/226536246 conn(0x7f32141001a0 msgr2=0x7f3214100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:02.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.211+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 shutdown_connections 2024-06-03T01:08:02.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:02.211+0000 7f321a000700 1 -- 172.21.15.94:0/226536246 wait complete. 2024-06-03T01:08:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:02 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/226536246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T01:08:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:02 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/226536246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T01:08:02.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:02 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/226536246' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-03T01:08:03.005 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:03.530 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037829 got 257698037832 for osd.8 2024-06-03T01:08:03.531 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:03.578 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:03 smithi094 ceph-mon[32119]: pgmap v434: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:08:03.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:03 smithi107 ceph-mon[36622]: pgmap v434: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:08:03.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:03 smithi179 ceph-mon[37041]: pgmap v434: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-03T01:08:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:04 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3621390492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T01:08:04.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462992 got 201863462995 for osd.6 2024-06-03T01:08:04.738 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:04 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3621390492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T01:08:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:04 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3621390492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-03T01:08:05.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.413+0000 7f76ad200700 1 -- 172.21.15.94:0/3181512388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a8103f00 msgr2=0x7f76a810c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:05.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.413+0000 7f76ad200700 1 --2- 172.21.15.94:0/3181512388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a8103f00 0x7f76a810c3f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7690009990 tx=0x7f7690009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.413+0000 7f76ad200700 1 -- 172.21.15.94:0/3181512388 shutdown_connections 2024-06-03T01:08:05.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.413+0000 7f76ad200700 1 --2- 172.21.15.94:0/3181512388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a810c9c0 0x7f76a810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.413+0000 7f76ad200700 1 --2- 172.21.15.94:0/3181512388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76a81035e0 0x7f76a81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.413+0000 7f76ad200700 1 --2- 172.21.15.94:0/3181512388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a8103f00 0x7f76a810c3f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.413+0000 7f76ad200700 1 -- 172.21.15.94:0/3181512388 >> 172.21.15.94:0/3181512388 conn(0x7f76a80fcdb0 msgr2=0x7f76a80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:05.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.414+0000 7f76ad200700 1 -- 172.21.15.94:0/3181512388 shutdown_connections 2024-06-03T01:08:05.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.414+0000 7f76ad200700 1 -- 172.21.15.94:0/3181512388 wait complete. 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.415+0000 7f76ad200700 1 Processor -- start 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.415+0000 7f76ad200700 1 -- start start 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76ad200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76a81035e0 0x7f76a819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76ad200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a8103f00 0x7f76a819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76ad200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a810c9c0 0x7f76a8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76ad200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8112730 con 0x7f76a810c9c0 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76ad200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81125b0 con 0x7f76a8103f00 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76ad200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81128b0 con 0x7f76a81035e0 2024-06-03T01:08:05.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76a7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a810c9c0 0x7f76a8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76a7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a810c9c0 0x7f76a8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33318/0 (socket says 172.21.15.94:33318) 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76a7e00700 1 -- 172.21.15.94:0/3330210305 learned_addr learned my addr 172.21.15.94:0/3330210305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f769fe00700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a8103f00 0x7f76a819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.416+0000 7f76a7400700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76a81035e0 0x7f76a819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a7400700 1 -- 172.21.15.94:0/3330210305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a8103f00 msgr2=0x7f76a819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a7400700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a8103f00 0x7f76a819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a7400700 1 -- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a810c9c0 msgr2=0x7f76a8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a7400700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a810c9c0 0x7f76a8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a7400700 1 -- 172.21.15.94:0/3330210305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a8199190 con 0x7f76a81035e0 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a7e00700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a810c9c0 0x7f76a8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:05.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f769fe00700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a8103f00 0x7f76a819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a7400700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76a81035e0 0x7f76a819e820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f769800e9f0 tx=0x7f769800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a4c00700 1 -- 172.21.15.94:0/3330210305 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698004220 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a4c00700 1 -- 172.21.15.94:0/3330210305 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7698004d10 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a8199480 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76a4c00700 1 -- 172.21.15.94:0/3330210305 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698005020 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.417+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a56f0 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.419+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a8069180 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.421+0000 7f76a4c00700 1 -- 172.21.15.94:0/3330210305 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7698004750 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.422+0000 7f76a4c00700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f768c0a2cb0 0x7f768c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.422+0000 7f76a4c00700 1 -- 172.21.15.94:0/3330210305 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f76980cf490 con 0x7f76a81035e0 2024-06-03T01:08:05.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.422+0000 7f769fe00700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f768c0a2cb0 0x7f768c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:05.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.423+0000 7f769fe00700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f768c0a2cb0 0x7f768c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76a810dcf0 tx=0x7f7690005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:05.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.426+0000 7f76a4c00700 1 -- 172.21.15.94:0/3330210305 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f769809beb0 con 0x7f76a81035e0 2024-06-03T01:08:05.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.661+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f76a8199f10 con 0x7f76a81035e0 2024-06-03T01:08:05.183 INFO:teuthology.orchestra.run.smithi094.stdout:292057776195 2024-06-03T01:08:05.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.662+0000 7f76a4c00700 1 -- 172.21.15.94:0/3330210305 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f76980043c0 con 0x7f76a81035e0 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f768c0a2cb0 msgr2=0x7f768c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f768c0a2cb0 0x7f768c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76a810dcf0 tx=0x7f7690005fd0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76a81035e0 msgr2=0x7f76a819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76a81035e0 0x7f76a819e820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f769800e9f0 tx=0x7f769800ed00 comp rx=0 tx=0).stop 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 shutdown_connections 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f768c0a2cb0 0x7f768c0a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76a810c9c0 0x7f76a8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76a81035e0 0x7f76a819e820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 --2- 172.21.15.94:0/3330210305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76a8103f00 0x7f76a819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:05.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 >> 172.21.15.94:0/3330210305 conn(0x7f76a80fcdb0 msgr2=0x7f76a80fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:05.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 shutdown_connections 2024-06-03T01:08:05.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:04.665+0000 7f76ad200700 1 -- 172.21.15.94:0/3330210305 wait complete. 2024-06-03T01:08:05.660 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:05 smithi094 ceph-mon[32119]: pgmap v435: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-03T01:08:05.660 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:05 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3330210305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T01:08:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:05 smithi107 ceph-mon[36622]: pgmap v435: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-03T01:08:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:05 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3330210305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T01:08:05.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:05 smithi179 ceph-mon[37041]: pgmap v435: 129 pgs: 129 active+clean; 651 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-03T01:08:05.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:05 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3330210305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-03T01:08:06.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299464 got 223338299470 for osd.7 2024-06-03T01:08:06.289 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:06.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776191 got 292057776195 for osd.9 2024-06-03T01:08:06.508 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:06.635 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:06 smithi094 ceph-mon[32119]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T01:08:06.635 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:06 smithi094 ceph-mon[32119]: pgmap v436: 129 pgs: 129 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T01:08:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:06 smithi107 ceph-mon[36622]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T01:08:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:06 smithi107 ceph-mon[36622]: pgmap v436: 129 pgs: 129 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T01:08:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:06 smithi179 ceph-mon[37041]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-03T01:08:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:06 smithi179 ceph-mon[37041]: pgmap v436: 129 pgs: 129 active+clean; 652 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.306+0000 7f1717a00700 1 -- 172.21.15.94:0/1818643135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17100746c0 msgr2=0x7f1710074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.306+0000 7f1717a00700 1 --2- 172.21.15.94:0/1818643135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17100746c0 0x7f1710074aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1700009f90 tx=0x7f170000b330 comp rx=0 tx=0).stop 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.307+0000 7f1717a00700 1 -- 172.21.15.94:0/1818643135 shutdown_connections 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.307+0000 7f1717a00700 1 --2- 172.21.15.94:0/1818643135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17100746c0 0x7f1710074aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.307+0000 7f1717a00700 1 --2- 172.21.15.94:0/1818643135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1710113660 0x7f1710115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.307+0000 7f1717a00700 1 --2- 172.21.15.94:0/1818643135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1710074fe0 0x7f1710113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.307+0000 7f1717a00700 1 -- 172.21.15.94:0/1818643135 >> 172.21.15.94:0/1818643135 conn(0x7f1710079640 msgr2=0x7f1710079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:07.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.307+0000 7f1717a00700 1 -- 172.21.15.94:0/1818643135 shutdown_connections 2024-06-03T01:08:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.307+0000 7f1717a00700 1 -- 172.21.15.94:0/1818643135 wait complete. 2024-06-03T01:08:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.308+0000 7f1717a00700 1 Processor -- start 2024-06-03T01:08:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.308+0000 7f1717a00700 1 -- start start 2024-06-03T01:08:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.308+0000 7f1717a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17100746c0 0x7f171019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.308+0000 7f1717a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1710074fe0 0x7f171019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.308+0000 7f1717a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1710113660 0x7f17101a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:07.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.308+0000 7f1717a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1710118df0 con 0x7f1710113660 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1717a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1710118c70 con 0x7f1710074fe0 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1717a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1710118f70 con 0x7f17100746c0 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17100746c0 0x7f171019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1710113660 0x7f17101a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17100746c0 0x7f171019f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49872/0 (socket says 172.21.15.94:49872) 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 -- 172.21.15.94:0/1005558505 learned_addr learned my addr 172.21.15.94:0/1005558505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1707e00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1710074fe0 0x7f171019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:07.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 -- 172.21.15.94:0/1005558505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1710074fe0 msgr2=0x7f171019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1710074fe0 0x7f171019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 -- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1710113660 msgr2=0x7f17101a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1710113660 0x7f17101a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.309+0000 7f1715000700 1 -- 172.21.15.94:0/1005558505 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17101a4310 con 0x7f17100746c0 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.310+0000 7f1715000700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17100746c0 0x7f171019f300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1700004f20 tx=0x7f1700004f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.310+0000 7f170e000700 1 -- 172.21.15.94:0/1005558505 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700021020 con 0x7f17100746c0 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.310+0000 7f170e000700 1 -- 172.21.15.94:0/1005558505 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1700007ac0 con 0x7f17100746c0 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.310+0000 7f170e000700 1 -- 172.21.15.94:0/1005558505 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17000052a0 con 0x7f17100746c0 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.311+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17101a45a0 con 0x7f17100746c0 2024-06-03T01:08:07.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.311+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17101abe80 con 0x7f17100746c0 2024-06-03T01:08:07.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.312+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1710075de0 con 0x7f17100746c0 2024-06-03T01:08:07.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.314+0000 7f170e000700 1 -- 172.21.15.94:0/1005558505 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1700024020 con 0x7f17100746c0 2024-06-03T01:08:07.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.315+0000 7f170e000700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f16f40a2ca0 0x7f16f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:07.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.316+0000 7f170e000700 1 -- 172.21.15.94:0/1005558505 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f170004fe00 con 0x7f17100746c0 2024-06-03T01:08:07.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.316+0000 7f1707e00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f16f40a2ca0 0x7f16f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:07.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.317+0000 7f1707e00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f16f40a2ca0 0x7f16f40a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009680 tx=0x7f16fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:07.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.318+0000 7f170e000700 1 -- 172.21.15.94:0/1005558505 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f170009d320 con 0x7f17100746c0 2024-06-03T01:08:07.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.556+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1710069180 con 0x7f17100746c0 2024-06-03T01:08:07.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.557+0000 7f170e000700 1 -- 172.21.15.94:0/1005558505 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f170009cc20 con 0x7f17100746c0 2024-06-03T01:08:07.557 INFO:teuthology.orchestra.run.smithi094.stdout:55834574961 2024-06-03T01:08:07.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f16f40a2ca0 msgr2=0x7f16f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:07.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f16f40a2ca0 0x7f16f40a5160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009680 tx=0x7f16fc009260 comp rx=0 tx=0).stop 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17100746c0 msgr2=0x7f171019f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17100746c0 0x7f171019f300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1700004f20 tx=0x7f1700004f50 comp rx=0 tx=0).stop 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 shutdown_connections 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f16f40a2ca0 0x7f16f40a5160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1710113660 0x7f17101a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17100746c0 0x7f171019f300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 --2- 172.21.15.94:0/1005558505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1710074fe0 0x7f171019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 >> 172.21.15.94:0/1005558505 conn(0x7f1710079640 msgr2=0x7f1710112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 shutdown_connections 2024-06-03T01:08:07.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.560+0000 7f1717a00700 1 -- 172.21.15.94:0/1005558505 wait complete. 2024-06-03T01:08:07.809 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:07 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1005558505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T01:08:08.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:07 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1005558505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T01:08:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:07 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1005558505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-03T01:08:08.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.402+0000 7f4103a00700 1 -- 172.21.15.94:0/3803830231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 msgr2=0x7f40fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.402+0000 7f4103a00700 1 --2- 172.21.15.94:0/3803830231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc074aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009990 tx=0x7f40f0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.402+0000 7f4103a00700 1 -- 172.21.15.94:0/3803830231 shutdown_connections 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.402+0000 7f4103a00700 1 --2- 172.21.15.94:0/3803830231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40fc113660 0x7f40fc115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.402+0000 7f4103a00700 1 --2- 172.21.15.94:0/3803830231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc074aa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.402+0000 7f4103a00700 1 --2- 172.21.15.94:0/3803830231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40fc074fe0 0x7f40fc113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.402+0000 7f4103a00700 1 -- 172.21.15.94:0/3803830231 >> 172.21.15.94:0/3803830231 conn(0x7f40fc0795f0 msgr2=0x7f40fc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.403+0000 7f4103a00700 1 -- 172.21.15.94:0/3803830231 shutdown_connections 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.403+0000 7f4103a00700 1 -- 172.21.15.94:0/3803830231 wait complete. 2024-06-03T01:08:08.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.404+0000 7f4103a00700 1 Processor -- start 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.404+0000 7f4103a00700 1 -- start start 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.404+0000 7f4103a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.404+0000 7f4103a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40fc074fe0 0x7f40fc19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.404+0000 7f4103a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40fc113660 0x7f40fc1a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f4103a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc118de0 con 0x7f40fc0746c0 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f4103a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc118c60 con 0x7f40fc113660 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f4103a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc118f60 con 0x7f40fc074fe0 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f4101000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f4101000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc19f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55548/0 (socket says 172.21.15.94:55548) 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f4101000700 1 -- 172.21.15.94:0/483441723 learned_addr learned my addr 172.21.15.94:0/483441723 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f40fbe00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40fc074fe0 0x7f40fc19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:08.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.405+0000 7f4101a00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40fc113660 0x7f40fc1a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40fbe00700 1 -- 172.21.15.94:0/483441723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40fc113660 msgr2=0x7f40fc1a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40fbe00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40fc113660 0x7f40fc1a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40fbe00700 1 -- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 msgr2=0x7f40fc19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40fbe00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40fbe00700 1 -- 172.21.15.94:0/483441723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc1a4340 con 0x7f40fc074fe0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f4101000700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40fbe00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40fc074fe0 0x7f40fc19f870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40f9600700 1 -- 172.21.15.94:0/483441723 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800cb70 con 0x7f40fc074fe0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc1a4630 con 0x7f40fc074fe0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40f9600700 1 -- 172.21.15.94:0/483441723 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e800ee90 con 0x7f40fc074fe0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc1abeb0 con 0x7f40fc074fe0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.406+0000 7f40f9600700 1 -- 172.21.15.94:0/483441723 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8003dc0 con 0x7f40fc074fe0 2024-06-03T01:08:08.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.408+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc005330 con 0x7f40fc074fe0 2024-06-03T01:08:08.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.409+0000 7f40f9600700 1 -- 172.21.15.94:0/483441723 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40e8004020 con 0x7f40fc074fe0 2024-06-03T01:08:08.577 INFO:teuthology.orchestra.run.smithi094.stdout:141733920864 2024-06-03T01:08:08.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.411+0000 7f40f9600700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40e40a2c20 0x7f40e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:08.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.411+0000 7f40f9600700 1 -- 172.21.15.94:0/483441723 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f40e80cf410 con 0x7f40fc074fe0 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.411+0000 7f4101000700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40e40a2c20 0x7f40e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.412+0000 7f4101000700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40e40a2c20 0x7f40e40a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009750 tx=0x7f40f000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.415+0000 7f40f9600700 1 -- 172.21.15.94:0/483441723 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f40e801e2e0 con 0x7f40fc074fe0 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.650+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f40dc005620 con 0x7f40fc074fe0 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.650+0000 7f40f9600700 1 -- 172.21.15.94:0/483441723 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f40e809bff0 con 0x7f40fc074fe0 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40e40a2c20 msgr2=0x7f40e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40e40a2c20 0x7f40e40a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009750 tx=0x7f40f000b280 comp rx=0 tx=0).stop 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40fc074fe0 msgr2=0x7f40fc19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40fc074fe0 0x7f40fc19f870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800ed00 comp rx=0 tx=0).stop 2024-06-03T01:08:08.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 shutdown_connections 2024-06-03T01:08:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40fc113660 0x7f40fc1a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f40e40a2c20 0x7f40e40a50e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40fc0746c0 0x7f40fc19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 --2- 172.21.15.94:0/483441723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40fc074fe0 0x7f40fc19f870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.653+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 >> 172.21.15.94:0/483441723 conn(0x7f40fc0795f0 msgr2=0x7f40fc1124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.654+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 shutdown_connections 2024-06-03T01:08:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:07.654+0000 7f4103a00700 1 -- 172.21.15.94:0/483441723 wait complete. 2024-06-03T01:08:08.846 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:08 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/483441723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T01:08:08.846 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:08 smithi094 ceph-mon[32119]: pgmap v437: 129 pgs: 129 active+clean; 652 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T01:08:09.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:08 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/483441723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T01:08:09.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:08 smithi107 ceph-mon[36622]: pgmap v437: 129 pgs: 129 active+clean; 652 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T01:08:09.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:08 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/483441723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-03T01:08:09.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:08 smithi179 ceph-mon[37041]: pgmap v437: 129 pgs: 129 active+clean; 652 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T01:08:09.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 -- 172.21.15.94:0/1914006221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 msgr2=0x7f0fac1115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 --2- 172.21.15.94:0/1914006221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 0x7f0fac1115a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40098f0 tx=0x7f0fa4009c00 comp rx=0 tx=0).stop 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 -- 172.21.15.94:0/1914006221 shutdown_connections 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 --2- 172.21.15.94:0/1914006221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 0x7f0fac1115a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 --2- 172.21.15.94:0/1914006221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fac10a6d0 0x7f0fac10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 --2- 172.21.15.94:0/1914006221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac1046d0 0x7f0fac104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 -- 172.21.15.94:0/1914006221 >> 172.21.15.94:0/1914006221 conn(0x7f0fac1001c0 msgr2=0x7f0fac1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 -- 172.21.15.94:0/1914006221 shutdown_connections 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.737+0000 7f0fb3200700 1 -- 172.21.15.94:0/1914006221 wait complete. 2024-06-03T01:08:09.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.738+0000 7f0fb3200700 1 Processor -- start 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.738+0000 7f0fb3200700 1 -- start start 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fb3200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fac1046d0 0x7f0fac19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fb3200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 0x7f0fac19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fb3200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac10a6d0 0x7f0fac19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fb3200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac114a60 con 0x7f0fac1046d0 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fb3200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1148e0 con 0x7f0fac10a6d0 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fb3200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac114be0 con 0x7f0fac105090 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fabe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fac1046d0 0x7f0fac19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fb1200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac10a6d0 0x7f0fac19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fabe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fac1046d0 0x7f0fac19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33290/0 (socket says 172.21.15.94:33290) 2024-06-03T01:08:09.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fabe00700 1 -- 172.21.15.94:0/3621390492 learned_addr learned my addr 172.21.15.94:0/3621390492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.739+0000 7f0fa3400700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 0x7f0fac19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb1200700 1 -- 172.21.15.94:0/3621390492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 msgr2=0x7f0fac19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb1200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 0x7f0fac19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb1200700 1 -- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fac1046d0 msgr2=0x7f0fac19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb1200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fac1046d0 0x7f0fac19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb1200700 1 -- 172.21.15.94:0/3621390492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac19fe60 con 0x7f0fac10a6d0 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb1200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac10a6d0 0x7f0fac19f720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009810 tx=0x7f0fa400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fa9600700 1 -- 172.21.15.94:0/3621390492 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4005140 con 0x7f0fac10a6d0 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac1a00f0 con 0x7f0fac10a6d0 2024-06-03T01:08:09.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fa9600700 1 -- 172.21.15.94:0/3621390492 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa400b560 con 0x7f0fac10a6d0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fa9600700 1 -- 172.21.15.94:0/3621390492 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4003a70 con 0x7f0fac10a6d0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.740+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac1a79d0 con 0x7f0fac10a6d0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.741+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fac105e10 con 0x7f0fac10a6d0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.743+0000 7f0fa9600700 1 -- 172.21.15.94:0/3621390492 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0fa400baa0 con 0x7f0fac10a6d0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.745+0000 7f0fa9600700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f900a2d30 0x7f0f900a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.745+0000 7f0fa9600700 1 -- 172.21.15.94:0/3621390492 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0fa40d07c0 con 0x7f0fac10a6d0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.745+0000 7f0fabe00700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f900a2d30 0x7f0f900a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.745+0000 7f0fabe00700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f900a2d30 0x7f0f900a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009c10 tx=0x7f0f9c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:09.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.747+0000 7f0fa9600700 1 -- 172.21.15.94:0/3621390492 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0fa405e850 con 0x7f0fac10a6d0 2024-06-03T01:08:09.225 INFO:teuthology.orchestra.run.smithi094.stdout:107374182500 2024-06-03T01:08:09.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.985+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0fac02dca0 con 0x7f0fac10a6d0 2024-06-03T01:08:09.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.986+0000 7f0fa9600700 1 -- 172.21.15.94:0/3621390492 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0fa4013030 con 0x7f0fac10a6d0 2024-06-03T01:08:09.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.988+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f900a2d30 msgr2=0x7f0f900a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.988+0000 7f0fb3200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f900a2d30 0x7f0f900a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009c10 tx=0x7f0f9c00b040 comp rx=0 tx=0).stop 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.988+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac10a6d0 msgr2=0x7f0fac19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.988+0000 7f0fb3200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac10a6d0 0x7f0fac19f720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009810 tx=0x7f0fa400fa70 comp rx=0 tx=0).stop 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 shutdown_connections 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac10a6d0 0x7f0fac19f720 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0f900a2d30 0x7f0f900a51f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fac1046d0 0x7f0fac19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 --2- 172.21.15.94:0/3621390492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fac105090 0x7f0fac19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:09.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 >> 172.21.15.94:0/3621390492 conn(0x7f0fac1001c0 msgr2=0x7f0fac1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:09.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 shutdown_connections 2024-06-03T01:08:09.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:03.989+0000 7f0fb3200700 1 -- 172.21.15.94:0/3621390492 wait complete. 2024-06-03T01:08:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:10.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.159+0000 7fbff7e00700 1 -- 172.21.15.94:0/183433614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0102740 msgr2=0x7fbff0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.159+0000 7fbff7e00700 1 --2- 172.21.15.94:0/183433614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0102740 0x7fbff0102b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400b0d0 tx=0x7fbfe400b3e0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.159+0000 7fbff7e00700 1 -- 172.21.15.94:0/183433614 shutdown_connections 2024-06-03T01:08:10.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.159+0000 7fbff7e00700 1 --2- 172.21.15.94:0/183433614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0107840 0x7fbff010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.159+0000 7fbff7e00700 1 --2- 172.21.15.94:0/183433614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0102740 0x7fbff0102b20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.159+0000 7fbff7e00700 1 --2- 172.21.15.94:0/183433614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbff01030f0 0x7fbff0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.159+0000 7fbff7e00700 1 -- 172.21.15.94:0/183433614 >> 172.21.15.94:0/183433614 conn(0x7fbff00795f0 msgr2=0x7fbff0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:10.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.160+0000 7fbff7e00700 1 -- 172.21.15.94:0/183433614 shutdown_connections 2024-06-03T01:08:10.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.160+0000 7fbff7e00700 1 -- 172.21.15.94:0/183433614 wait complete. 2024-06-03T01:08:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 Processor -- start 2024-06-03T01:08:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 -- start start 2024-06-03T01:08:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0102740 0x7fbff0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbff01030f0 0x7fbff0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0107840 0x7fbff0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01127d0 con 0x7fbff0107840 2024-06-03T01:08:10.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0112650 con 0x7fbff01030f0 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.161+0000 7fbff7e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0112950 con 0x7fbff0102740 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0107840 0x7fbff0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0102740 0x7fbff0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0102740 0x7fbff0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49898/0 (socket says 172.21.15.94:49898) 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 -- 172.21.15.94:0/541732893 learned_addr learned my addr 172.21.15.94:0/541732893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff4a00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbff01030f0 0x7fbff0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 -- 172.21.15.94:0/541732893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbff01030f0 msgr2=0x7fbff0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbff01030f0 0x7fbff0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 -- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0107840 msgr2=0x7fbff0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0107840 0x7fbff0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.162+0000 7fbff5400700 1 -- 172.21.15.94:0/541732893 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff01a7170 con 0x7fbff0102740 2024-06-03T01:08:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.163+0000 7fbff5400700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0102740 0x7fbff0074790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4007a20 tx=0x7fbfe4007a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.163+0000 7fbfee000700 1 -- 172.21.15.94:0/541732893 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4021020 con 0x7fbff0102740 2024-06-03T01:08:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.163+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff01a7400 con 0x7fbff0102740 2024-06-03T01:08:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.163+0000 7fbfee000700 1 -- 172.21.15.94:0/541732893 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe4004c50 con 0x7fbff0102740 2024-06-03T01:08:10.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.163+0000 7fbfee000700 1 -- 172.21.15.94:0/541732893 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe40052b0 con 0x7fbff0102740 2024-06-03T01:08:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.163+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01a7900 con 0x7fbff0102740 2024-06-03T01:08:10.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.164+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff010aa80 con 0x7fbff0102740 2024-06-03T01:08:10.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.166+0000 7fbfee000700 1 -- 172.21.15.94:0/541732893 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbfe4024020 con 0x7fbff0102740 2024-06-03T01:08:10.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.167+0000 7fbfee000700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbfd80a2c20 0x7fbfd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.168+0000 7fbfee000700 1 -- 172.21.15.94:0/541732893 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fbfe400f020 con 0x7fbff0102740 2024-06-03T01:08:10.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.168+0000 7fbff4a00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbfd80a2c20 0x7fbfd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.168+0000 7fbff4a00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbfd80a2c20 0x7fbfd80a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbff0107270 tx=0x7fbfdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.170+0000 7fbfee000700 1 -- 172.21.15.94:0/541732893 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbfe409d0b0 con 0x7fbff0102740 2024-06-03T01:08:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.396+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbff010a460 con 0x7fbff0102740 2024-06-03T01:08:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.397+0000 7fbfee000700 1 -- 172.21.15.94:0/541732893 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfe409c9b0 con 0x7fbff0102740 2024-06-03T01:08:10.398 INFO:teuthology.orchestra.run.smithi094.stdout:34359738486 2024-06-03T01:08:10.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbfd80a2c20 msgr2=0x7fbfd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbfd80a2c20 0x7fbfd80a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbff0107270 tx=0x7fbfdc009260 comp rx=0 tx=0).stop 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0102740 msgr2=0x7fbff0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0102740 0x7fbff0074790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4007a20 tx=0x7fbfe4007a50 comp rx=0 tx=0).stop 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 shutdown_connections 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fbfd80a2c20 0x7fbfd80a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbff0107840 0x7fbff0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbff0102740 0x7fbff0074790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 --2- 172.21.15.94:0/541732893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbff01030f0 0x7fbff0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.401+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 >> 172.21.15.94:0/541732893 conn(0x7fbff00795f0 msgr2=0x7fbff0108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:10.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.402+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 shutdown_connections 2024-06-03T01:08:10.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.402+0000 7fbff7e00700 1 -- 172.21.15.94:0/541732893 wait complete. 2024-06-03T01:08:10.572 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574954 got 55834574961 for osd.1 2024-06-03T01:08:10.572 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:10.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:10 smithi094 ceph-mon[32119]: pgmap v438: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T01:08:10.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:10 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/541732893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T01:08:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:10 smithi107 ceph-mon[36622]: pgmap v438: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T01:08:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:10 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/541732893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T01:08:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:10 smithi179 ceph-mon[37041]: pgmap v438: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-03T01:08:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:10 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/541732893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-03T01:08:10.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 -- 172.21.15.94:0/2605443684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 msgr2=0x7f0f08104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2605443684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f08104a40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009990 tx=0x7f0ef4009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 -- 172.21.15.94:0/2605443684 shutdown_connections 2024-06-03T01:08:10.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2605443684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f0810a670 0x7f0f0810aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2605443684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f08104f80 0x7f0f08111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2605443684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f08104a40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 -- 172.21.15.94:0/2605443684 >> 172.21.15.94:0/2605443684 conn(0x7f0f081000d0 msgr2=0x7f0f081024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 -- 172.21.15.94:0/2605443684 shutdown_connections 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.787+0000 7f0f07e00700 1 -- 172.21.15.94:0/2605443684 wait complete. 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.788+0000 7f0f07e00700 1 Processor -- start 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.788+0000 7f0f07e00700 1 -- start start 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f07e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f081a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f07e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f08104f80 0x7f0f081a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f07e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f0810a670 0x7f0f0819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f07e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08114920 con 0x7f0f08104f80 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f07e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081147a0 con 0x7f0f0810a670 2024-06-03T01:08:10.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f07e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08114aa0 con 0x7f0f081045c0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f06a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f081a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f06000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f08104f80 0x7f0f081a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f06a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f081a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49922/0 (socket says 172.21.15.94:49922) 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.789+0000 7f0f06a00700 1 -- 172.21.15.94:0/2191715555 learned_addr learned my addr 172.21.15.94:0/2191715555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.790+0000 7f0f07400700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f0810a670 0x7f0f0819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.790+0000 7f0f06a00700 1 -- 172.21.15.94:0/2191715555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f0810a670 msgr2=0x7f0f0819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.790+0000 7f0f06a00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f0810a670 0x7f0f0819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.790+0000 7f0f06a00700 1 -- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f08104f80 msgr2=0x7f0f081a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.790+0000 7f0f06a00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f08104f80 0x7f0f081a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.790+0000 7f0f06a00700 1 -- 172.21.15.94:0/2191715555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0819b200 con 0x7f0f081045c0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.790+0000 7f0f06a00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f081a08e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800e9a0 tx=0x7f0ef800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.791+0000 7f0f0ca00700 1 -- 172.21.15.94:0/2191715555 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef800ca10 con 0x7f0f081045c0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.791+0000 7f0f0ca00700 1 -- 172.21.15.94:0/2191715555 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef800cbb0 con 0x7f0f081045c0 2024-06-03T01:08:10.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.791+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0819b4f0 con 0x7f0f081045c0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.791+0000 7f0f0ca00700 1 -- 172.21.15.94:0/2191715555 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8018680 con 0x7f0f081045c0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.791+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081a77b0 con 0x7f0f081045c0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.793+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f08069180 con 0x7f0f081045c0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.795+0000 7f0f0ca00700 1 -- 172.21.15.94:0/2191715555 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ef8010430 con 0x7f0f081045c0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.797+0000 7f0f0ca00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0eec0a2c20 0x7f0eec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.797+0000 7f0f0ca00700 1 -- 172.21.15.94:0/2191715555 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f0ef80cf600 con 0x7f0f081045c0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.797+0000 7f0f06000700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0eec0a2c20 0x7f0eec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.798+0000 7f0f06000700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0eec0a2c20 0x7f0eec0a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400b2e0 tx=0x7f0ef4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:10.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:10.801+0000 7f0f0ca00700 1 -- 172.21.15.94:0/2191715555 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0ef809c1e0 con 0x7f0f081045c0 2024-06-03T01:08:11.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.032+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0f081a7980 con 0x7f0f081045c0 2024-06-03T01:08:11.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.033+0000 7f0f0ca00700 1 -- 172.21.15.94:0/2191715555 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ef809bae0 con 0x7f0f081045c0 2024-06-03T01:08:11.034 INFO:teuthology.orchestra.run.smithi094.stdout:167503724635 2024-06-03T01:08:11.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0eec0a2c20 msgr2=0x7f0eec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0eec0a2c20 0x7f0eec0a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400b2e0 tx=0x7f0ef4005960 comp rx=0 tx=0).stop 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 msgr2=0x7f0f081a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f081a08e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800e9a0 tx=0x7f0ef800ecb0 comp rx=0 tx=0).stop 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 shutdown_connections 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f0810a670 0x7f0f0819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f0eec0a2c20 0x7f0eec0a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f081045c0 0x7f0f081a08e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 --2- 172.21.15.94:0/2191715555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f08104f80 0x7f0f081a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.036+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 >> 172.21.15.94:0/2191715555 conn(0x7f0f081000d0 msgr2=0x7f0f08101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.037+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 shutdown_connections 2024-06-03T01:08:11.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:11.037+0000 7f0f07e00700 1 -- 172.21.15.94:0/2191715555 wait complete. 2024-06-03T01:08:11.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:11 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2191715555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T01:08:11.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:11 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2191715555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T01:08:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:11 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2191715555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-03T01:08:12.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:12 smithi094 ceph-mon[32119]: pgmap v439: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T01:08:12.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:12 smithi107 ceph-mon[36622]: pgmap v439: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T01:08:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:12 smithi179 ceph-mon[37041]: pgmap v439: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-03T01:08:12.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182493 got 107374182500 for osd.3 2024-06-03T01:08:12.937 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:13.336 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920857 got 141733920864 for osd.4 2024-06-03T01:08:13.336 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:14.239 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738479 got 34359738486 for osd.0 2024-06-03T01:08:14.239 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 -- 172.21.15.94:0/1323019983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 msgr2=0x7f56181088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 --2- 172.21.15.94:0/1323019983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 0x7f56181088d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5608009990 tx=0x7f5608009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 -- 172.21.15.94:0/1323019983 shutdown_connections 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 --2- 172.21.15.94:0/1323019983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f561806b3c0 0x7f561810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 --2- 172.21.15.94:0/1323019983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 0x7f56181088d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 --2- 172.21.15.94:0/1323019983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561806aa00 0x7f561806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 -- 172.21.15.94:0/1323019983 >> 172.21.15.94:0/1323019983 conn(0x7f5618078300 msgr2=0x7f5618078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:14.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.449+0000 7f561e800700 1 -- 172.21.15.94:0/1323019983 shutdown_connections 2024-06-03T01:08:14.451 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:14 smithi094 ceph-mon[32119]: pgmap v440: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-03T01:08:14.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.450+0000 7f561e800700 1 -- 172.21.15.94:0/1323019983 wait complete. 2024-06-03T01:08:14.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 Processor -- start 2024-06-03T01:08:14.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 -- start start 2024-06-03T01:08:14.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561806aa00 0x7f5618198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:14.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f561806b3c0 0x7f5618199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:14.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 0x7f561819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:14.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56181127c0 con 0x7f56181084f0 2024-06-03T01:08:14.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5618112640 con 0x7f561806b3c0 2024-06-03T01:08:14.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.451+0000 7f561e800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5618112940 con 0x7f561806aa00 2024-06-03T01:08:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5617400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561806aa00 0x7f5618198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5617400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561806aa00 0x7f5618198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49940/0 (socket says 172.21.15.94:49940) 2024-06-03T01:08:14.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5617400700 1 -- 172.21.15.94:0/2606022613 learned_addr learned my addr 172.21.15.94:0/2606022613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5617e00700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 0x7f561819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5616a00700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f561806b3c0 0x7f5618199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5616a00700 1 -- 172.21.15.94:0/2606022613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561806aa00 msgr2=0x7f5618198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5616a00700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561806aa00 0x7f5618198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5616a00700 1 -- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 msgr2=0x7f561819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5616a00700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 0x7f561819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5616a00700 1 -- 172.21.15.94:0/2606022613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561819de00 con 0x7f561806b3c0 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.452+0000 7f5617e00700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 0x7f561819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.453+0000 7f5616a00700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f561806b3c0 0x7f5618199330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f560400e9f0 tx=0x7f560400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.453+0000 7f560fe00700 1 -- 172.21.15.94:0/2606022613 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604004220 con 0x7f561806b3c0 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.453+0000 7f560fe00700 1 -- 172.21.15.94:0/2606022613 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5604004d10 con 0x7f561806b3c0 2024-06-03T01:08:14.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.453+0000 7f560fe00700 1 -- 172.21.15.94:0/2606022613 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604005020 con 0x7f561806b3c0 2024-06-03T01:08:14.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.453+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561819e0f0 con 0x7f561806b3c0 2024-06-03T01:08:14.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.453+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56181a5970 con 0x7f561806b3c0 2024-06-03T01:08:14.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.455+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561810a9d0 con 0x7f561806b3c0 2024-06-03T01:08:14.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.457+0000 7f560fe00700 1 -- 172.21.15.94:0/2606022613 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5604004750 con 0x7f561806b3c0 2024-06-03T01:08:14.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.458+0000 7f560fe00700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f55fc0a2c20 0x7f55fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:14.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.458+0000 7f560fe00700 1 -- 172.21.15.94:0/2606022613 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f56040cf2d0 con 0x7f561806b3c0 2024-06-03T01:08:14.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.459+0000 7f5617400700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f55fc0a2c20 0x7f55fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:14.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.459+0000 7f5617400700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f55fc0a2c20 0x7f55fc0a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5608009750 tx=0x7f560800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:14.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.463+0000 7f560fe00700 1 -- 172.21.15.94:0/2606022613 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f560409be50 con 0x7f561806b3c0 2024-06-03T01:08:14.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.692+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f561819a150 con 0x7f561806b3c0 2024-06-03T01:08:14.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.693+0000 7f560fe00700 1 -- 172.21.15.94:0/2606022613 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f560409b750 con 0x7f561806b3c0 2024-06-03T01:08:14.693 INFO:teuthology.orchestra.run.smithi094.stdout:77309411437 2024-06-03T01:08:14.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.695+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f55fc0a2c20 msgr2=0x7f55fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:14.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.695+0000 7f561e800700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f55fc0a2c20 0x7f55fc0a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5608009750 tx=0x7f560800b280 comp rx=0 tx=0).stop 2024-06-03T01:08:14.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.695+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f561806b3c0 msgr2=0x7f5618199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:14.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.695+0000 7f561e800700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f561806b3c0 0x7f5618199330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f560400e9f0 tx=0x7f560400ed00 comp rx=0 tx=0).stop 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 shutdown_connections 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f55fc0a2c20 0x7f55fc0a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56181084f0 0x7f561819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f561806aa00 0x7f5618198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 --2- 172.21.15.94:0/2606022613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f561806b3c0 0x7f5618199330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 >> 172.21.15.94:0/2606022613 conn(0x7f5618078300 msgr2=0x7f5618100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 shutdown_connections 2024-06-03T01:08:14.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:14.696+0000 7f561e800700 1 -- 172.21.15.94:0/2606022613 wait complete. 2024-06-03T01:08:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:14 smithi107 ceph-mon[36622]: pgmap v440: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-03T01:08:14.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:14 smithi179 ceph-mon[37041]: pgmap v440: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-03T01:08:15.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:15 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2606022613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T01:08:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:15 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2606022613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T01:08:15.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:15 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2606022613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-03T01:08:15.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411430 got 77309411437 for osd.2 2024-06-03T01:08:15.918 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:16.482 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:16 smithi094 ceph-mon[32119]: pgmap v441: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-03T01:08:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:16 smithi107 ceph-mon[36622]: pgmap v441: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-03T01:08:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:16 smithi179 ceph-mon[37041]: pgmap v441: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-06-03T01:08:17.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 -- 172.21.15.94:0/807004365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0108ac0 msgr2=0x7f74a0108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:17.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 --2- 172.21.15.94:0/807004365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0108ac0 0x7f74a0108f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f748c0098f0 tx=0x7f748c009c00 comp rx=0 tx=0).stop 2024-06-03T01:08:17.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 -- 172.21.15.94:0/807004365 shutdown_connections 2024-06-03T01:08:17.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 --2- 172.21.15.94:0/807004365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0109480 0x7f74a0115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 --2- 172.21.15.94:0/807004365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74a010eac0 0x7f74a010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 --2- 172.21.15.94:0/807004365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0108ac0 0x7f74a0108f40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 -- 172.21.15.94:0/807004365 >> 172.21.15.94:0/807004365 conn(0x7f74a0075c60 msgr2=0x7f74a01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 -- 172.21.15.94:0/807004365 shutdown_connections 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.646+0000 7f74a8600700 1 -- 172.21.15.94:0/807004365 wait complete. 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 Processor -- start 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 -- start start 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0108ac0 0x7f74a019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0109480 0x7f74a019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74a010eac0 0x7f74a01a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0118e50 con 0x7f74a0108ac0 2024-06-03T01:08:17.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0118cd0 con 0x7f74a0109480 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a8600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0118fd0 con 0x7f74a010eac0 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.648+0000 7f74a5c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0108ac0 0x7f74a019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0108ac0 0x7f74a019f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53772/0 (socket says 172.21.15.94:53772) 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a6600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74a010eac0 0x7f74a01a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 -- 172.21.15.94:0/1015798149 learned_addr learned my addr 172.21.15.94:0/1015798149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5200700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0109480 0x7f74a019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 -- 172.21.15.94:0/1015798149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74a010eac0 msgr2=0x7f74a01a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74a010eac0 0x7f74a01a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 -- 172.21.15.94:0/1015798149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0109480 msgr2=0x7f74a019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0109480 0x7f74a019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 -- 172.21.15.94:0/1015798149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01a42b0 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a6600700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74a010eac0 0x7f74a01a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5200700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0109480 0x7f74a019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.649+0000 7f74a5c00700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0108ac0 0x7f74a019f2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f749400ec10 tx=0x7f749400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.650+0000 7f749e000700 1 -- 172.21.15.94:0/1015798149 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749400cc80 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.650+0000 7f749e000700 1 -- 172.21.15.94:0/1015798149 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7494004500 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.650+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01a45a0 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.650+0000 7f749e000700 1 -- 172.21.15.94:0/1015798149 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494010690 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.650+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01abe20 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.651+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a0069180 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.653+0000 7f749e000700 1 -- 172.21.15.94:0/1015798149 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7494003e50 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.655+0000 7f749e000700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f74880a2c20 0x7f74880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.655+0000 7f749e000700 1 -- 172.21.15.94:0/1015798149 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f74940cf680 con 0x7f74a0108ac0 2024-06-03T01:08:17.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.655+0000 7f74a5200700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f74880a2c20 0x7f74880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stdout:313532612671 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.656+0000 7f74a5200700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f74880a2c20 0x7f74880a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f748c0098c0 tx=0x7f748c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.660+0000 7f749e000700 1 -- 172.21.15.94:0/1015798149 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f749409c200 con 0x7f74a0108ac0 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.883+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f74a01a0600 con 0x7f74a0108ac0 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.884+0000 7f749e000700 1 -- 172.21.15.94:0/1015798149 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f749400ce20 con 0x7f74a0108ac0 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.887+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f74880a2c20 msgr2=0x7f74880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.887+0000 7f74a8600700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f74880a2c20 0x7f74880a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f748c0098c0 tx=0x7f748c019040 comp rx=0 tx=0).stop 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.887+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0108ac0 msgr2=0x7f74a019f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:17.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.887+0000 7f74a8600700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0108ac0 0x7f74a019f2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f749400ec10 tx=0x7f749400ef20 comp rx=0 tx=0).stop 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 shutdown_connections 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f74880a2c20 0x7f74880a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74a010eac0 0x7f74a01a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74a0108ac0 0x7f74a019f2a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 --2- 172.21.15.94:0/1015798149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a0109480 0x7f74a019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 >> 172.21.15.94:0/1015798149 conn(0x7f74a0075c60 msgr2=0x7f74a0076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 shutdown_connections 2024-06-03T01:08:17.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:16.888+0000 7f74a8600700 1 -- 172.21.15.94:0/1015798149 wait complete. 2024-06-03T01:08:17.739 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:17 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1015798149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T01:08:17.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:17 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1015798149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T01:08:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:17 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1015798149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-03T01:08:18.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724628 got 167503724635 for osd.5 2024-06-03T01:08:18.328 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:18 smithi094 ceph-mon[32119]: pgmap v442: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T01:08:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:18 smithi107 ceph-mon[36622]: pgmap v442: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T01:08:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:18 smithi179 ceph-mon[37041]: pgmap v442: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-03T01:08:18.939 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612663 got 313532612671 for osd.10 2024-06-03T01:08:18.939 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:19.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.596+0000 7f04f8600700 1 -- 172.21.15.94:0/3662198844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0107840 msgr2=0x7f04f010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:19.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.596+0000 7f04f8600700 1 --2- 172.21.15.94:0/3662198844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0107840 0x7f04f010f3c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04e800b0d0 tx=0x7f04e800b3e0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.597+0000 7f04f8600700 1 -- 172.21.15.94:0/3662198844 shutdown_connections 2024-06-03T01:08:19.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.597+0000 7f04f8600700 1 --2- 172.21.15.94:0/3662198844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04f0102740 0x7f04f0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.597+0000 7f04f8600700 1 --2- 172.21.15.94:0/3662198844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0107840 0x7f04f010f3c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.597+0000 7f04f8600700 1 --2- 172.21.15.94:0/3662198844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f01030f0 0x7f04f0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.597+0000 7f04f8600700 1 -- 172.21.15.94:0/3662198844 >> 172.21.15.94:0/3662198844 conn(0x7f04f00795f0 msgr2=0x7f04f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:19.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.597+0000 7f04f8600700 1 -- 172.21.15.94:0/3662198844 shutdown_connections 2024-06-03T01:08:19.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.597+0000 7f04f8600700 1 -- 172.21.15.94:0/3662198844 wait complete. 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 Processor -- start 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 -- start start 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0102740 0x7f04f019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f01030f0 0x7f04f019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04f0107840 0x7f04f019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01127b0 con 0x7f04f0102740 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0112630 con 0x7f04f0107840 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.598+0000 7f04f8600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0112930 con 0x7f04f01030f0 2024-06-03T01:08:19.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f5c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0102740 0x7f04f019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:19.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f5c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0102740 0x7f04f019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53786/0 (socket says 172.21.15.94:53786) 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f5c00700 1 -- 172.21.15.94:0/1899285477 learned_addr learned my addr 172.21.15.94:0/1899285477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f6600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04f0107840 0x7f04f019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f5200700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f01030f0 0x7f04f019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f6600700 1 -- 172.21.15.94:0/1899285477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f01030f0 msgr2=0x7f04f019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f6600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f01030f0 0x7f04f019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f6600700 1 -- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0102740 msgr2=0x7f04f019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f6600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0102740 0x7f04f019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f6600700 1 -- 172.21.15.94:0/1899285477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f019fdb0 con 0x7f04f0107840 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.599+0000 7f04f5c00700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0102740 0x7f04f019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.600+0000 7f04f6600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04f0107840 0x7f04f019f670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04e800d010 tx=0x7f04e8009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.600+0000 7f04ee000700 1 -- 172.21.15.94:0/1899285477 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8003a00 con 0x7f04f0107840 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.600+0000 7f04ee000700 1 -- 172.21.15.94:0/1899285477 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e8022070 con 0x7f04f0107840 2024-06-03T01:08:19.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.600+0000 7f04ee000700 1 -- 172.21.15.94:0/1899285477 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e801b6c0 con 0x7f04f0107840 2024-06-03T01:08:19.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.600+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f01a0040 con 0x7f04f0107840 2024-06-03T01:08:19.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.600+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a7920 con 0x7f04f0107840 2024-06-03T01:08:19.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.601+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f0103bd0 con 0x7f04f0107840 2024-06-03T01:08:19.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.604+0000 7f04ee000700 1 -- 172.21.15.94:0/1899285477 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04e8003ba0 con 0x7f04f0107840 2024-06-03T01:08:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.605+0000 7f04ee000700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f04d80a2c20 0x7f04d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.605+0000 7f04ee000700 1 -- 172.21.15.94:0/1899285477 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f04e80d02b0 con 0x7f04f0107840 2024-06-03T01:08:19.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.605+0000 7f04f5c00700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f04d80a2c20 0x7f04d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:19.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.606+0000 7f04f5c00700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f04d80a2c20 0x7f04d80a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04e4011550 tx=0x7f04e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:19.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.607+0000 7f04ee000700 1 -- 172.21.15.94:0/1899285477 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04e809cec0 con 0x7f04f0107840 2024-06-03T01:08:19.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.833+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f04f01a7af0 con 0x7f04f0107840 2024-06-03T01:08:19.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.834+0000 7f04ee000700 1 -- 172.21.15.94:0/1899285477 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f04e809c7c0 con 0x7f04f0107840 2024-06-03T01:08:19.835 INFO:teuthology.orchestra.run.smithi094.stdout:347892351034 2024-06-03T01:08:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f04d80a2c20 msgr2=0x7f04d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f04d80a2c20 0x7f04d80a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04e4011550 tx=0x7f04e4009260 comp rx=0 tx=0).stop 2024-06-03T01:08:19.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04f0107840 msgr2=0x7f04f019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04f0107840 0x7f04f019f670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04e800d010 tx=0x7f04e8009810 comp rx=0 tx=0).stop 2024-06-03T01:08:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 shutdown_connections 2024-06-03T01:08:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04f0107840 0x7f04f019f670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f04d80a2c20 0x7f04d80a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04f0102740 0x7f04f019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 --2- 172.21.15.94:0/1899285477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04f01030f0 0x7f04f019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 >> 172.21.15.94:0/1899285477 conn(0x7f04f00795f0 msgr2=0x7f04f0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.837+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 shutdown_connections 2024-06-03T01:08:19.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:19.838+0000 7f04f8600700 1 -- 172.21.15.94:0/1899285477 wait complete. 2024-06-03T01:08:20.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:19 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1899285477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T01:08:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:19 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1899285477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T01:08:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:19 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1899285477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-03T01:08:20.751 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351028 got 347892351034 for osd.11 2024-06-03T01:08:20.751 DEBUG:teuthology.parallel:result is None 2024-06-03T01:08:20.751 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-03T01:08:20.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T01:08:21.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:20 smithi094 ceph-mon[32119]: pgmap v443: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-06-03T01:08:21.197 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:21.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:20 smithi107 ceph-mon[36622]: pgmap v443: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-06-03T01:08:21.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:20 smithi179 ceph-mon[37041]: pgmap v443: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-06-03T01:08:22.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:22 smithi094 ceph-mon[32119]: pgmap v444: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:22 smithi107 ceph-mon[36622]: pgmap v444: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:22.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:22 smithi179 ceph-mon[37041]: pgmap v444: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:23.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.595+0000 7f6038e00700 1 -- 172.21.15.94:0/619208382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 msgr2=0x7f603410e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:23.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.595+0000 7f6038e00700 1 --2- 172.21.15.94:0/619208382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603410e950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6020009990 tx=0x7f6020009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.595+0000 7f6038e00700 1 -- 172.21.15.94:0/619208382 shutdown_connections 2024-06-03T01:08:23.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.595+0000 7f6038e00700 1 --2- 172.21.15.94:0/619208382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f603410ee90 0x7f6034111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.595+0000 7f6038e00700 1 --2- 172.21.15.94:0/619208382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60340746c0 0x7f6034074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.595+0000 7f6038e00700 1 --2- 172.21.15.94:0/619208382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603410e950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.595+0000 7f6038e00700 1 -- 172.21.15.94:0/619208382 >> 172.21.15.94:0/619208382 conn(0x7f6034079530 msgr2=0x7f6034079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:23.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.596+0000 7f6038e00700 1 -- 172.21.15.94:0/619208382 shutdown_connections 2024-06-03T01:08:23.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.596+0000 7f6038e00700 1 -- 172.21.15.94:0/619208382 wait complete. 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.597+0000 7f6038e00700 1 Processor -- start 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.597+0000 7f6038e00700 1 -- start start 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6038e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60340746c0 0x7f603419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6038e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6038e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f603410ee90 0x7f603419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6038e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60341149f0 con 0x7f6034074fe0 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6038e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6034114870 con 0x7f603410ee90 2024-06-03T01:08:23.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6038e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6034114b70 con 0x7f60340746c0 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53808/0 (socket says 172.21.15.94:53808) 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031400700 1 -- 172.21.15.94:0/1450231508 learned_addr learned my addr 172.21.15.94:0/1450231508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60340746c0 0x7f603419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6032800700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f603410ee90 0x7f603419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031400700 1 -- 172.21.15.94:0/1450231508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60340746c0 msgr2=0x7f603419af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031400700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60340746c0 0x7f603419af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031400700 1 -- 172.21.15.94:0/1450231508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f603410ee90 msgr2=0x7f603419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:23.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.598+0000 7f6031400700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f603410ee90 0x7f603419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f6031400700 1 -- 172.21.15.94:0/1450231508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603419fec0 con 0x7f6034074fe0 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f6031e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60340746c0 0x7f603419af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f6031400700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603419b440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6020000c00 tx=0x7f602000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f602aa00700 1 -- 172.21.15.94:0/1450231508 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020027070 con 0x7f6034074fe0 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f602aa00700 1 -- 172.21.15.94:0/1450231508 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60200056d0 con 0x7f6034074fe0 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60341a0150 con 0x7f6034074fe0 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a7a30 con 0x7f6034074fe0 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.599+0000 7f602aa00700 1 -- 172.21.15.94:0/1450231508 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020031060 con 0x7f6034074fe0 2024-06-03T01:08:23.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.600+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6034069180 con 0x7f6034074fe0 2024-06-03T01:08:23.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.602+0000 7f602aa00700 1 -- 172.21.15.94:0/1450231508 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f602002a070 con 0x7f6034074fe0 2024-06-03T01:08:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.604+0000 7f602aa00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f60180a2c20 0x7f60180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.604+0000 7f602aa00700 1 -- 172.21.15.94:0/1450231508 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f60200cffe0 con 0x7f6034074fe0 2024-06-03T01:08:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.604+0000 7f6031e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f60180a2c20 0x7f60180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:23.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.605+0000 7f6031e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f60180a2c20 0x7f60180a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6024011490 tx=0x7f6024009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:23.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.608+0000 7f602aa00700 1 -- 172.21.15.94:0/1450231508 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6020013070 con 0x7f6034074fe0 2024-06-03T01:08:23.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.827+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f603419c210 con 0x7f60180a2c20 2024-06-03T01:08:23.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.840+0000 7f602aa00700 1 -- 172.21.15.94:0/1450231508 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340725 (secure 0 0 0) 0x7f603419c210 con 0x7f60180a2c20 2024-06-03T01:08:23.849 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f60180a2c20 msgr2=0x7f60180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f60180a2c20 0x7f60180a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6024011490 tx=0x7f6024009260 comp rx=0 tx=0).stop 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 msgr2=0x7f603419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603419b440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6020000c00 tx=0x7f602000bab0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 shutdown_connections 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f603410ee90 0x7f603419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f60180a2c20 0x7f60180a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60340746c0 0x7f603419af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 --2- 172.21.15.94:0/1450231508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6034074fe0 0x7f603419b440 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:23.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.847+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 >> 172.21.15.94:0/1450231508 conn(0x7f6034079530 msgr2=0x7f60340ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:23.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.848+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 shutdown_connections 2024-06-03T01:08:23.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:23.848+0000 7f6038e00700 1 -- 172.21.15.94:0/1450231508 wait complete. 2024-06-03T01:08:23.853 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-06-03T01:08:24.728 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:24 smithi094 ceph-mon[32119]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:08:24.729 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:24 smithi094 ceph-mon[32119]: pgmap v445: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:24.732 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-06-03T01:08:22.351768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667448,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1408,"num_read_kb":843,"num_write":449,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":410652,"kb_used_data":15740,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124531172,"statfs":{"total":1151940427776,"available":1151519920128,"internally_reserved":0,"allocated":16117760,"data_stored":8869166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":404272442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006576"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288498+0000","last_change":"2024-06-03T01:04:54.382830+0000","last_active":"2024-06-03T01:08:04.288498+0000","last_peered":"2024-06-03T01:08:04.288498+0000","last_clean":"2024-06-03T01:08:04.288498+0000","last_became_active":"2024-06-03T01:04:54.382328+0000","last_became_peered":"2024-06-03T01:04:54.382328+0000","last_unstale":"2024-06-03T01:08:04.288498+0000","last_undegraded":"2024-06-03T01:08:04.288498+0000","last_fullsized":"2024-06-03T01:08:04.288498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:59.037188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288252+0000","last_change":"2024-06-03T01:04:54.380179+0000","last_active":"2024-06-03T01:08:04.288252+0000","last_peered":"2024-06-03T01:08:04.288252+0000","last_clean":"2024-06-03T01:08:04.288252+0000","last_became_active":"2024-06-03T01:04:54.370975+0000","last_became_peered":"2024-06-03T01:04:54.370975+0000","last_unstale":"2024-06-03T01:08:04.288252+0000","last_undegraded":"2024-06-03T01:08:04.288252+0000","last_fullsized":"2024-06-03T01:08:04.288252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:41.133053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290162+0000","last_change":"2024-06-03T01:04:54.381423+0000","last_active":"2024-06-03T01:08:04.290162+0000","last_peered":"2024-06-03T01:08:04.290162+0000","last_clean":"2024-06-03T01:08:04.290162+0000","last_became_active":"2024-06-03T01:04:54.375659+0000","last_became_peered":"2024-06-03T01:04:54.375659+0000","last_unstale":"2024-06-03T01:08:04.290162+0000","last_undegraded":"2024-06-03T01:08:04.290162+0000","last_fullsized":"2024-06-03T01:08:04.290162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:50.294425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.258917+0000","last_change":"2024-06-03T01:04:54.372494+0000","last_active":"2024-06-03T01:08:04.258917+0000","last_peered":"2024-06-03T01:08:04.258917+0000","last_clean":"2024-06-03T01:08:04.258917+0000","last_became_active":"2024-06-03T01:04:54.371744+0000","last_became_peered":"2024-06-03T01:04:54.371744+0000","last_unstale":"2024-06-03T01:08:04.258917+0000","last_undegraded":"2024-06-03T01:08:04.258917+0000","last_fullsized":"2024-06-03T01:08:04.258917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:01.887751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.272884+0000","last_change":"2024-06-03T01:04:54.394675+0000","last_active":"2024-06-03T01:08:04.272884+0000","last_peered":"2024-06-03T01:08:04.272884+0000","last_clean":"2024-06-03T01:08:04.272884+0000","last_became_active":"2024-06-03T01:04:54.394246+0000","last_became_peered":"2024-06-03T01:04:54.394246+0000","last_unstale":"2024-06-03T01:08:04.272884+0000","last_undegraded":"2024-06-03T01:08:04.272884+0000","last_fullsized":"2024-06-03T01:08:04.272884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:48:47.331598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.264227+0000","last_change":"2024-06-03T01:04:54.378765+0000","last_active":"2024-06-03T01:08:04.264227+0000","last_peered":"2024-06-03T01:08:04.264227+0000","last_clean":"2024-06-03T01:08:04.264227+0000","last_became_active":"2024-06-03T01:04:54.378306+0000","last_became_peered":"2024-06-03T01:04:54.378306+0000","last_unstale":"2024-06-03T01:08:04.264227+0000","last_undegraded":"2024-06-03T01:08:04.264227+0000","last_fullsized":"2024-06-03T01:08:04.264227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:26:20.321196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.278742+0000","last_change":"2024-06-03T01:05:35.252114+0000","last_active":"2024-06-03T01:08:04.278742+0000","last_peered":"2024-06-03T01:08:04.278742+0000","last_clean":"2024-06-03T01:08:04.278742+0000","last_became_active":"2024-06-03T01:05:35.251849+0000","last_became_peered":"2024-06-03T01:05:35.251849+0000","last_unstale":"2024-06-03T01:08:04.278742+0000","last_undegraded":"2024-06-03T01:08:04.278742+0000","last_fullsized":"2024-06-03T01:08:04.278742+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:18.942723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.274537+0000","last_change":"2024-06-03T01:04:54.397423+0000","last_active":"2024-06-03T01:08:04.274537+0000","last_peered":"2024-06-03T01:08:04.274537+0000","last_clean":"2024-06-03T01:08:04.274537+0000","last_became_active":"2024-06-03T01:04:54.394329+0000","last_became_peered":"2024-06-03T01:04:54.394329+0000","last_unstale":"2024-06-03T01:08:04.274537+0000","last_undegraded":"2024-06-03T01:08:04.274537+0000","last_fullsized":"2024-06-03T01:08:04.274537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:04.020364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.260269+0000","last_change":"2024-06-03T01:04:54.368589+0000","last_active":"2024-06-03T01:08:04.260269+0000","last_peered":"2024-06-03T01:08:04.260269+0000","last_clean":"2024-06-03T01:08:04.260269+0000","last_became_active":"2024-06-03T01:04:54.368068+0000","last_became_peered":"2024-06-03T01:04:54.368068+0000","last_unstale":"2024-06-03T01:08:04.260269+0000","last_undegraded":"2024-06-03T01:08:04.260269+0000","last_fullsized":"2024-06-03T01:08:04.260269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:59:44.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.258394+0000","last_change":"2024-06-03T01:04:54.367488+0000","last_active":"2024-06-03T01:08:04.258394+0000","last_peered":"2024-06-03T01:08:04.258394+0000","last_clean":"2024-06-03T01:08:04.258394+0000","last_became_active":"2024-06-03T01:04:54.367022+0000","last_became_peered":"2024-06-03T01:04:54.367022+0000","last_unstale":"2024-06-03T01:08:04.258394+0000","last_undegraded":"2024-06-03T01:08:04.258394+0000","last_fullsized":"2024-06-03T01:08:04.258394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:52:27.697948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.268971+0000","last_change":"2024-06-03T01:04:54.366790+0000","last_active":"2024-06-03T01:08:04.268971+0000","last_peered":"2024-06-03T01:08:04.268971+0000","last_clean":"2024-06-03T01:08:04.268971+0000","last_became_active":"2024-06-03T01:04:54.366456+0000","last_became_peered":"2024-06-03T01:04:54.366456+0000","last_unstale":"2024-06-03T01:08:04.268971+0000","last_undegraded":"2024-06-03T01:08:04.268971+0000","last_fullsized":"2024-06-03T01:08:04.268971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:15:42.495284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.264768+0000","last_change":"2024-06-03T01:04:54.369483+0000","last_active":"2024-06-03T01:08:04.264768+0000","last_peered":"2024-06-03T01:08:04.264768+0000","last_clean":"2024-06-03T01:08:04.264768+0000","last_became_active":"2024-06-03T01:04:54.369118+0000","last_became_peered":"2024-06-03T01:04:54.369118+0000","last_unstale":"2024-06-03T01:08:04.264768+0000","last_undegraded":"2024-06-03T01:08:04.264768+0000","last_fullsized":"2024-06-03T01:08:04.264768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:18.218426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.285401+0000","last_change":"2024-06-03T01:04:54.380648+0000","last_active":"2024-06-03T01:08:04.285401+0000","last_peered":"2024-06-03T01:08:04.285401+0000","last_clean":"2024-06-03T01:08:04.285401+0000","last_became_active":"2024-06-03T01:04:54.380195+0000","last_became_peered":"2024-06-03T01:04:54.380195+0000","last_unstale":"2024-06-03T01:08:04.285401+0000","last_undegraded":"2024-06-03T01:08:04.285401+0000","last_fullsized":"2024-06-03T01:08:04.285401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:04.991858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290318+0000","last_change":"2024-06-03T01:04:54.367250+0000","last_active":"2024-06-03T01:08:04.290318+0000","last_peered":"2024-06-03T01:08:04.290318+0000","last_clean":"2024-06-03T01:08:04.290318+0000","last_became_active":"2024-06-03T01:04:54.364817+0000","last_became_peered":"2024-06-03T01:04:54.364817+0000","last_unstale":"2024-06-03T01:08:04.290318+0000","last_undegraded":"2024-06-03T01:08:04.290318+0000","last_fullsized":"2024-06-03T01:08:04.290318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:55:20.948692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.291050+0000","last_change":"2024-06-03T01:04:54.367640+0000","last_active":"2024-06-03T01:08:04.291050+0000","last_peered":"2024-06-03T01:08:04.291050+0000","last_clean":"2024-06-03T01:08:04.291050+0000","last_became_active":"2024-06-03T01:04:54.367080+0000","last_became_peered":"2024-06-03T01:04:54.367080+0000","last_unstale":"2024-06-03T01:08:04.291050+0000","last_undegraded":"2024-06-03T01:08:04.291050+0000","last_fullsized":"2024-06-03T01:08:04.291050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:43:23.469092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.293903+0000","last_change":"2024-06-03T01:04:54.376236+0000","last_active":"2024-06-03T01:08:04.293903+0000","last_peered":"2024-06-03T01:08:04.293903+0000","last_clean":"2024-06-03T01:08:04.293903+0000","last_became_active":"2024-06-03T01:04:54.375866+0000","last_became_peered":"2024-06-03T01:04:54.375866+0000","last_unstale":"2024-06-03T01:08:04.293903+0000","last_undegraded":"2024-06-03T01:08:04.293903+0000","last_fullsized":"2024-06-03T01:08:04.293903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:45.926391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.291713+0000","last_change":"2024-06-03T01:04:54.388824+0000","last_active":"2024-06-03T01:08:04.291713+0000","last_peered":"2024-06-03T01:08:04.291713+0000","last_clean":"2024-06-03T01:08:04.291713+0000","last_became_active":"2024-06-03T01:04:54.388491+0000","last_became_peered":"2024-06-03T01:04:54.388491+0000","last_unstale":"2024-06-03T01:08:04.291713+0000","last_undegraded":"2024-06-03T01:08:04.291713+0000","last_fullsized":"2024-06-03T01:08:04.291713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:48.593126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.270633+0000","last_change":"2024-06-03T01:04:54.377623+0000","last_active":"2024-06-03T01:08:04.270633+0000","last_peered":"2024-06-03T01:08:04.270633+0000","last_clean":"2024-06-03T01:08:04.270633+0000","last_became_active":"2024-06-03T01:04:54.377037+0000","last_became_peered":"2024-06-03T01:04:54.377037+0000","last_unstale":"2024-06-03T01:08:04.270633+0000","last_undegraded":"2024-06-03T01:08:04.270633+0000","last_fullsized":"2024-06-03T01:08:04.270633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:52:12.729604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.261154+0000","last_change":"2024-06-03T01:04:54.396946+0000","last_active":"2024-06-03T01:08:04.261154+0000","last_peered":"2024-06-03T01:08:04.261154+0000","last_clean":"2024-06-03T01:08:04.261154+0000","last_became_active":"2024-06-03T01:04:54.396695+0000","last_became_peered":"2024-06-03T01:04:54.396695+0000","last_unstale":"2024-06-03T01:08:04.261154+0000","last_undegraded":"2024-06-03T01:08:04.261154+0000","last_fullsized":"2024-06-03T01:08:04.261154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:10:57.842403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347515+0000","last_change":"2024-06-03T01:05:08.399118+0000","last_active":"2024-06-03T01:06:35.347515+0000","last_peered":"2024-06-03T01:06:35.347515+0000","last_clean":"2024-06-03T01:06:35.347515+0000","last_became_active":"2024-06-03T01:05:08.398716+0000","last_became_peered":"2024-06-03T01:05:08.398716+0000","last_unstale":"2024-06-03T01:06:35.347515+0000","last_undegraded":"2024-06-03T01:06:35.347515+0000","last_fullsized":"2024-06-03T01:06:35.347515+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:28:46.911683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'14","reported_seq":181,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:18.609728+0000","last_change":"2024-06-03T01:04:49.781140+0000","last_active":"2024-06-03T01:08:18.609728+0000","last_peered":"2024-06-03T01:08:18.609728+0000","last_clean":"2024-06-03T01:08:18.609728+0000","last_became_active":"2024-06-03T01:04:49.780712+0000","last_became_peered":"2024-06-03T01:04:49.780712+0000","last_unstale":"2024-06-03T01:08:18.609728+0000","last_undegraded":"2024-06-03T01:08:18.609728+0000","last_fullsized":"2024-06-03T01:08:18.609728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:12:48.170997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.266137+0000","last_change":"2024-06-03T01:04:54.396470+0000","last_active":"2024-06-03T01:08:04.266137+0000","last_peered":"2024-06-03T01:08:04.266137+0000","last_clean":"2024-06-03T01:08:04.266137+0000","last_became_active":"2024-06-03T01:04:54.394564+0000","last_became_peered":"2024-06-03T01:04:54.394564+0000","last_unstale":"2024-06-03T01:08:04.266137+0000","last_undegraded":"2024-06-03T01:08:04.266137+0000","last_fullsized":"2024-06-03T01:08:04.266137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:21:06.436992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354702+0000","last_change":"2024-06-03T01:05:08.410302+0000","last_active":"2024-06-03T01:06:35.354702+0000","last_peered":"2024-06-03T01:06:35.354702+0000","last_clean":"2024-06-03T01:06:35.354702+0000","last_became_active":"2024-06-03T01:05:08.409971+0000","last_became_peered":"2024-06-03T01:05:08.409971+0000","last_unstale":"2024-06-03T01:06:35.354702+0000","last_undegraded":"2024-06-03T01:06:35.354702+0000","last_fullsized":"2024-06-03T01:06:35.354702+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:50.786491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841198+0000","last_change":"2024-06-03T01:04:49.893771+0000","last_active":"2024-06-03T01:06:35.841198+0000","last_peered":"2024-06-03T01:06:35.841198+0000","last_clean":"2024-06-03T01:06:35.841198+0000","last_became_active":"2024-06-03T01:04:49.892855+0000","last_became_peered":"2024-06-03T01:04:49.892855+0000","last_unstale":"2024-06-03T01:06:35.841198+0000","last_undegraded":"2024-06-03T01:06:35.841198+0000","last_fullsized":"2024-06-03T01:06:35.841198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:47.474318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.296471+0000","last_change":"2024-06-03T01:04:54.378309+0000","last_active":"2024-06-03T01:08:04.296471+0000","last_peered":"2024-06-03T01:08:04.296471+0000","last_clean":"2024-06-03T01:08:04.296471+0000","last_became_active":"2024-06-03T01:04:54.377926+0000","last_became_peered":"2024-06-03T01:04:54.377926+0000","last_unstale":"2024-06-03T01:08:04.296471+0000","last_undegraded":"2024-06-03T01:08:04.296471+0000","last_fullsized":"2024-06-03T01:08:04.296471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:14:48.919904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.842782+0000","last_change":"2024-06-03T01:06:35.842264+0000","last_active":"2024-06-03T01:06:35.842782+0000","last_peered":"2024-06-03T01:06:35.842782+0000","last_clean":"2024-06-03T01:06:35.842782+0000","last_became_active":"2024-06-03T01:06:35.841786+0000","last_became_peered":"2024-06-03T01:06:35.841786+0000","last_unstale":"2024-06-03T01:06:35.842782+0000","last_undegraded":"2024-06-03T01:06:35.842782+0000","last_fullsized":"2024-06-03T01:06:35.842782+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:15.788405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838007+0000","last_change":"2024-06-03T01:04:49.770978+0000","last_active":"2024-06-03T01:06:35.838007+0000","last_peered":"2024-06-03T01:06:35.838007+0000","last_clean":"2024-06-03T01:06:35.838007+0000","last_became_active":"2024-06-03T01:04:49.770010+0000","last_became_peered":"2024-06-03T01:04:49.770010+0000","last_unstale":"2024-06-03T01:06:35.838007+0000","last_undegraded":"2024-06-03T01:06:35.838007+0000","last_fullsized":"2024-06-03T01:06:35.838007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:33.707218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.276969+0000","last_change":"2024-06-03T01:04:54.380616+0000","last_active":"2024-06-03T01:08:04.276969+0000","last_peered":"2024-06-03T01:08:04.276969+0000","last_clean":"2024-06-03T01:08:04.276969+0000","last_became_active":"2024-06-03T01:04:54.380257+0000","last_became_peered":"2024-06-03T01:04:54.380257+0000","last_unstale":"2024-06-03T01:08:04.276969+0000","last_undegraded":"2024-06-03T01:08:04.276969+0000","last_fullsized":"2024-06-03T01:08:04.276969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:14:54.327146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354695+0000","last_change":"2024-06-03T01:05:08.388139+0000","last_active":"2024-06-03T01:06:35.354695+0000","last_peered":"2024-06-03T01:06:35.354695+0000","last_clean":"2024-06-03T01:06:35.354695+0000","last_became_active":"2024-06-03T01:05:08.387697+0000","last_became_peered":"2024-06-03T01:05:08.387697+0000","last_unstale":"2024-06-03T01:06:35.354695+0000","last_undegraded":"2024-06-03T01:06:35.354695+0000","last_fullsized":"2024-06-03T01:06:35.354695+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:36:44.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837437+0000","last_change":"2024-06-03T01:04:49.771928+0000","last_active":"2024-06-03T01:06:35.837437+0000","last_peered":"2024-06-03T01:06:35.837437+0000","last_clean":"2024-06-03T01:06:35.837437+0000","last_became_active":"2024-06-03T01:04:49.771517+0000","last_became_peered":"2024-06-03T01:04:49.771517+0000","last_unstale":"2024-06-03T01:06:35.837437+0000","last_undegraded":"2024-06-03T01:06:35.837437+0000","last_fullsized":"2024-06-03T01:06:35.837437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:48:48.147276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.281239+0000","last_change":"2024-06-03T01:05:35.261543+0000","last_active":"2024-06-03T01:08:04.281239+0000","last_peered":"2024-06-03T01:08:04.281239+0000","last_clean":"2024-06-03T01:08:04.281239+0000","last_became_active":"2024-06-03T01:05:35.260977+0000","last_became_peered":"2024-06-03T01:05:35.260977+0000","last_unstale":"2024-06-03T01:08:04.281239+0000","last_undegraded":"2024-06-03T01:08:04.281239+0000","last_fullsized":"2024-06-03T01:08:04.281239+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:09:41.748477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.348031+0000","last_change":"2024-06-03T01:05:08.387864+0000","last_active":"2024-06-03T01:06:35.348031+0000","last_peered":"2024-06-03T01:06:35.348031+0000","last_clean":"2024-06-03T01:06:35.348031+0000","last_became_active":"2024-06-03T01:05:08.387381+0000","last_became_peered":"2024-06-03T01:05:08.387381+0000","last_unstale":"2024-06-03T01:06:35.348031+0000","last_undegraded":"2024-06-03T01:06:35.348031+0000","last_fullsized":"2024-06-03T01:06:35.348031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:31.349917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":177,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:18.509539+0000","last_change":"2024-06-03T01:04:49.770745+0000","last_active":"2024-06-03T01:08:18.509539+0000","last_peered":"2024-06-03T01:08:18.509539+0000","last_clean":"2024-06-03T01:08:18.509539+0000","last_became_active":"2024-06-03T01:04:49.770011+0000","last_became_peered":"2024-06-03T01:04:49.770011+0000","last_unstale":"2024-06-03T01:08:18.509539+0000","last_undegraded":"2024-06-03T01:08:18.509539+0000","last_fullsized":"2024-06-03T01:08:18.509539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:58:30.702656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.257080+0000","last_change":"2024-06-03T01:04:54.378551+0000","last_active":"2024-06-03T01:08:04.257080+0000","last_peered":"2024-06-03T01:08:04.257080+0000","last_clean":"2024-06-03T01:08:04.257080+0000","last_became_active":"2024-06-03T01:04:54.378201+0000","last_became_peered":"2024-06-03T01:04:54.378201+0000","last_unstale":"2024-06-03T01:08:04.257080+0000","last_undegraded":"2024-06-03T01:08:04.257080+0000","last_fullsized":"2024-06-03T01:08:04.257080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:39:25.918280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347887+0000","last_change":"2024-06-03T01:05:08.407416+0000","last_active":"2024-06-03T01:06:35.347887+0000","last_peered":"2024-06-03T01:06:35.347887+0000","last_clean":"2024-06-03T01:06:35.347887+0000","last_became_active":"2024-06-03T01:05:08.406968+0000","last_became_peered":"2024-06-03T01:05:08.406968+0000","last_unstale":"2024-06-03T01:06:35.347887+0000","last_undegraded":"2024-06-03T01:06:35.347887+0000","last_fullsized":"2024-06-03T01:06:35.347887+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:35.618908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363873+0000","last_change":"2024-06-03T01:04:49.772292+0000","last_active":"2024-06-03T01:06:35.363873+0000","last_peered":"2024-06-03T01:06:35.363873+0000","last_clean":"2024-06-03T01:06:35.363873+0000","last_became_active":"2024-06-03T01:04:49.771964+0000","last_became_peered":"2024-06-03T01:04:49.771964+0000","last_unstale":"2024-06-03T01:06:35.363873+0000","last_undegraded":"2024-06-03T01:06:35.363873+0000","last_fullsized":"2024-06-03T01:06:35.363873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:45.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.272773+0000","last_change":"2024-06-03T01:04:54.393864+0000","last_active":"2024-06-03T01:08:04.272773+0000","last_peered":"2024-06-03T01:08:04.272773+0000","last_clean":"2024-06-03T01:08:04.272773+0000","last_became_active":"2024-06-03T01:04:54.393442+0000","last_became_peered":"2024-06-03T01:04:54.393442+0000","last_unstale":"2024-06-03T01:08:04.272773+0000","last_undegraded":"2024-06-03T01:08:04.272773+0000","last_fullsized":"2024-06-03T01:08:04.272773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:29.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.599718+0000","last_change":"2024-06-03T01:06:35.599230+0000","last_active":"2024-06-03T01:06:35.599718+0000","last_peered":"2024-06-03T01:06:35.599718+0000","last_clean":"2024-06-03T01:06:35.599718+0000","last_became_active":"2024-06-03T01:06:35.598811+0000","last_became_peered":"2024-06-03T01:06:35.598811+0000","last_unstale":"2024-06-03T01:06:35.599718+0000","last_undegraded":"2024-06-03T01:06:35.599718+0000","last_fullsized":"2024-06-03T01:06:35.599718+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:50.943560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350527+0000","last_change":"2024-06-03T01:04:49.781942+0000","last_active":"2024-06-03T01:06:35.350527+0000","last_peered":"2024-06-03T01:06:35.350527+0000","last_clean":"2024-06-03T01:06:35.350527+0000","last_became_active":"2024-06-03T01:04:49.781585+0000","last_became_peered":"2024-06-03T01:04:49.781585+0000","last_unstale":"2024-06-03T01:06:35.350527+0000","last_undegraded":"2024-06-03T01:06:35.350527+0000","last_fullsized":"2024-06-03T01:06:35.350527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:06:07.814196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.277836+0000","last_change":"2024-06-03T01:04:54.390073+0000","last_active":"2024-06-03T01:08:04.277836+0000","last_peered":"2024-06-03T01:08:04.277836+0000","last_clean":"2024-06-03T01:08:04.277836+0000","last_became_active":"2024-06-03T01:04:54.389698+0000","last_became_peered":"2024-06-03T01:04:54.389698+0000","last_unstale":"2024-06-03T01:08:04.277836+0000","last_undegraded":"2024-06-03T01:08:04.277836+0000","last_fullsized":"2024-06-03T01:08:04.277836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:17:26.447432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597337+0000","last_change":"2024-06-03T01:05:08.395848+0000","last_active":"2024-06-03T01:06:35.597337+0000","last_peered":"2024-06-03T01:06:35.597337+0000","last_clean":"2024-06-03T01:06:35.597337+0000","last_became_active":"2024-06-03T01:05:08.394960+0000","last_became_peered":"2024-06-03T01:05:08.394960+0000","last_unstale":"2024-06-03T01:06:35.597337+0000","last_undegraded":"2024-06-03T01:06:35.597337+0000","last_fullsized":"2024-06-03T01:06:35.597337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:15.567354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366719+0000","last_change":"2024-06-03T01:04:49.770855+0000","last_active":"2024-06-03T01:06:35.366719+0000","last_peered":"2024-06-03T01:06:35.366719+0000","last_clean":"2024-06-03T01:06:35.366719+0000","last_became_active":"2024-06-03T01:04:49.770553+0000","last_became_peered":"2024-06-03T01:04:49.770553+0000","last_unstale":"2024-06-03T01:06:35.366719+0000","last_undegraded":"2024-06-03T01:06:35.366719+0000","last_fullsized":"2024-06-03T01:06:35.366719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:57.663765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.283973+0000","last_change":"2024-06-03T01:04:54.395962+0000","last_active":"2024-06-03T01:08:04.283973+0000","last_peered":"2024-06-03T01:08:04.283973+0000","last_clean":"2024-06-03T01:08:04.283973+0000","last_became_active":"2024-06-03T01:04:54.395463+0000","last_became_peered":"2024-06-03T01:04:54.395463+0000","last_unstale":"2024-06-03T01:08:04.283973+0000","last_undegraded":"2024-06-03T01:08:04.283973+0000","last_fullsized":"2024-06-03T01:08:04.283973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:48.570980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839998+0000","last_change":"2024-06-03T01:05:08.392196+0000","last_active":"2024-06-03T01:06:35.839998+0000","last_peered":"2024-06-03T01:06:35.839998+0000","last_clean":"2024-06-03T01:06:35.839998+0000","last_became_active":"2024-06-03T01:05:08.388980+0000","last_became_peered":"2024-06-03T01:05:08.388980+0000","last_unstale":"2024-06-03T01:06:35.839998+0000","last_undegraded":"2024-06-03T01:06:35.839998+0000","last_fullsized":"2024-06-03T01:06:35.839998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:01:01.066423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'6","reported_seq":299,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:21.513128+0000","last_change":"2024-06-03T01:04:49.785677+0000","last_active":"2024-06-03T01:08:21.513128+0000","last_peered":"2024-06-03T01:08:21.513128+0000","last_clean":"2024-06-03T01:08:21.513128+0000","last_became_active":"2024-06-03T01:04:49.785306+0000","last_became_peered":"2024-06-03T01:04:49.785306+0000","last_unstale":"2024-06-03T01:08:21.513128+0000","last_undegraded":"2024-06-03T01:08:21.513128+0000","last_fullsized":"2024-06-03T01:08:21.513128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:04:51.861728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":248,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.272212+0000","last_change":"2024-06-03T01:04:54.370886+0000","last_active":"2024-06-03T01:08:04.272212+0000","last_peered":"2024-06-03T01:08:04.272212+0000","last_clean":"2024-06-03T01:08:04.272212+0000","last_became_active":"2024-06-03T01:04:54.370555+0000","last_became_peered":"2024-06-03T01:04:54.370555+0000","last_unstale":"2024-06-03T01:08:04.272212+0000","last_undegraded":"2024-06-03T01:08:04.272212+0000","last_fullsized":"2024-06-03T01:08:04.272212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:20:23.802391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351608+0000","last_change":"2024-06-03T01:05:08.405735+0000","last_active":"2024-06-03T01:06:35.351608+0000","last_peered":"2024-06-03T01:06:35.351608+0000","last_clean":"2024-06-03T01:06:35.351608+0000","last_became_active":"2024-06-03T01:05:08.405289+0000","last_became_peered":"2024-06-03T01:05:08.405289+0000","last_unstale":"2024-06-03T01:06:35.351608+0000","last_undegraded":"2024-06-03T01:06:35.351608+0000","last_fullsized":"2024-06-03T01:06:35.351608+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:47:52.099470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382518+0000","last_change":"2024-06-03T01:04:49.771708+0000","last_active":"2024-06-03T01:06:35.382518+0000","last_peered":"2024-06-03T01:06:35.382518+0000","last_clean":"2024-06-03T01:06:35.382518+0000","last_became_active":"2024-06-03T01:04:49.771303+0000","last_became_peered":"2024-06-03T01:04:49.771303+0000","last_unstale":"2024-06-03T01:06:35.382518+0000","last_undegraded":"2024-06-03T01:06:35.382518+0000","last_fullsized":"2024-06-03T01:06:35.382518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:15.307453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.295154+0000","last_change":"2024-06-03T01:04:54.381206+0000","last_active":"2024-06-03T01:08:04.295154+0000","last_peered":"2024-06-03T01:08:04.295154+0000","last_clean":"2024-06-03T01:08:04.295154+0000","last_became_active":"2024-06-03T01:04:54.380842+0000","last_became_peered":"2024-06-03T01:04:54.380842+0000","last_unstale":"2024-06-03T01:08:04.295154+0000","last_undegraded":"2024-06-03T01:08:04.295154+0000","last_fullsized":"2024-06-03T01:08:04.295154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:37.487354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355269+0000","last_change":"2024-06-03T01:05:08.402554+0000","last_active":"2024-06-03T01:06:35.355269+0000","last_peered":"2024-06-03T01:06:35.355269+0000","last_clean":"2024-06-03T01:06:35.355269+0000","last_became_active":"2024-06-03T01:05:08.402040+0000","last_became_peered":"2024-06-03T01:05:08.402040+0000","last_unstale":"2024-06-03T01:06:35.355269+0000","last_undegraded":"2024-06-03T01:06:35.355269+0000","last_fullsized":"2024-06-03T01:06:35.355269+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:10.713793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366984+0000","last_change":"2024-06-03T01:04:49.895941+0000","last_active":"2024-06-03T01:06:35.366984+0000","last_peered":"2024-06-03T01:06:35.366984+0000","last_clean":"2024-06-03T01:06:35.366984+0000","last_became_active":"2024-06-03T01:04:49.895579+0000","last_became_peered":"2024-06-03T01:04:49.895579+0000","last_unstale":"2024-06-03T01:06:35.366984+0000","last_undegraded":"2024-06-03T01:06:35.366984+0000","last_fullsized":"2024-06-03T01:06:35.366984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:45:31.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.264473+0000","last_change":"2024-06-03T01:04:54.369271+0000","last_active":"2024-06-03T01:08:04.264473+0000","last_peered":"2024-06-03T01:08:04.264473+0000","last_clean":"2024-06-03T01:08:04.264473+0000","last_became_active":"2024-06-03T01:04:54.368896+0000","last_became_peered":"2024-06-03T01:04:54.368896+0000","last_unstale":"2024-06-03T01:08:04.264473+0000","last_undegraded":"2024-06-03T01:08:04.264473+0000","last_fullsized":"2024-06-03T01:08:04.264473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:47:54.079642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351058+0000","last_change":"2024-06-03T01:05:08.406394+0000","last_active":"2024-06-03T01:06:35.351058+0000","last_peered":"2024-06-03T01:06:35.351058+0000","last_clean":"2024-06-03T01:06:35.351058+0000","last_became_active":"2024-06-03T01:05:08.405356+0000","last_became_peered":"2024-06-03T01:05:08.405356+0000","last_unstale":"2024-06-03T01:06:35.351058+0000","last_undegraded":"2024-06-03T01:06:35.351058+0000","last_fullsized":"2024-06-03T01:06:35.351058+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:57:40.648759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'42","reported_seq":345,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:21.512960+0000","last_change":"2024-06-03T01:04:49.771275+0000","last_active":"2024-06-03T01:08:21.512960+0000","last_peered":"2024-06-03T01:08:21.512960+0000","last_clean":"2024-06-03T01:08:21.512960+0000","last_became_active":"2024-06-03T01:04:49.770750+0000","last_became_peered":"2024-06-03T01:04:49.770750+0000","last_unstale":"2024-06-03T01:08:21.512960+0000","last_undegraded":"2024-06-03T01:08:21.512960+0000","last_fullsized":"2024-06-03T01:08:21.512960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:27:01.251134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68831,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":258,"num_read_kb":197,"num_write":44,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.257623+0000","last_change":"2024-06-03T01:04:54.364655+0000","last_active":"2024-06-03T01:08:04.257623+0000","last_peered":"2024-06-03T01:08:04.257623+0000","last_clean":"2024-06-03T01:08:04.257623+0000","last_became_active":"2024-06-03T01:04:54.364309+0000","last_became_peered":"2024-06-03T01:04:54.364309+0000","last_unstale":"2024-06-03T01:08:04.257623+0000","last_undegraded":"2024-06-03T01:08:04.257623+0000","last_fullsized":"2024-06-03T01:08:04.257623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:18.141328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.367298+0000","last_change":"2024-06-03T01:05:08.393014+0000","last_active":"2024-06-03T01:06:35.367298+0000","last_peered":"2024-06-03T01:06:35.367298+0000","last_clean":"2024-06-03T01:06:35.367298+0000","last_became_active":"2024-06-03T01:05:08.392614+0000","last_became_peered":"2024-06-03T01:05:08.392614+0000","last_unstale":"2024-06-03T01:06:35.367298+0000","last_undegraded":"2024-06-03T01:06:35.367298+0000","last_fullsized":"2024-06-03T01:06:35.367298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:36.625154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347784+0000","last_change":"2024-06-03T01:04:49.778551+0000","last_active":"2024-06-03T01:06:35.347784+0000","last_peered":"2024-06-03T01:06:35.347784+0000","last_clean":"2024-06-03T01:06:35.347784+0000","last_became_active":"2024-06-03T01:04:49.778166+0000","last_became_peered":"2024-06-03T01:04:49.778166+0000","last_unstale":"2024-06-03T01:06:35.347784+0000","last_undegraded":"2024-06-03T01:06:35.347784+0000","last_fullsized":"2024-06-03T01:06:35.347784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:02:40.207601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.273743+0000","last_change":"2024-06-03T01:04:54.394722+0000","last_active":"2024-06-03T01:08:04.273743+0000","last_peered":"2024-06-03T01:08:04.273743+0000","last_clean":"2024-06-03T01:08:04.273743+0000","last_became_active":"2024-06-03T01:04:54.393548+0000","last_became_peered":"2024-06-03T01:04:54.393548+0000","last_unstale":"2024-06-03T01:08:04.273743+0000","last_undegraded":"2024-06-03T01:08:04.273743+0000","last_fullsized":"2024-06-03T01:08:04.273743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:49.077880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354625+0000","last_change":"2024-06-03T01:05:08.403334+0000","last_active":"2024-06-03T01:06:35.354625+0000","last_peered":"2024-06-03T01:06:35.354625+0000","last_clean":"2024-06-03T01:06:35.354625+0000","last_became_active":"2024-06-03T01:05:08.402835+0000","last_became_peered":"2024-06-03T01:05:08.402835+0000","last_unstale":"2024-06-03T01:06:35.354625+0000","last_undegraded":"2024-06-03T01:06:35.354625+0000","last_fullsized":"2024-06-03T01:06:35.354625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:50:08.551919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366736+0000","last_change":"2024-06-03T01:04:49.782860+0000","last_active":"2024-06-03T01:06:35.366736+0000","last_peered":"2024-06-03T01:06:35.366736+0000","last_clean":"2024-06-03T01:06:35.366736+0000","last_became_active":"2024-06-03T01:04:49.782610+0000","last_became_peered":"2024-06-03T01:04:49.782610+0000","last_unstale":"2024-06-03T01:06:35.366736+0000","last_undegraded":"2024-06-03T01:06:35.366736+0000","last_fullsized":"2024-06-03T01:06:35.366736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:37.967304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.285685+0000","last_change":"2024-06-03T01:05:35.253841+0000","last_active":"2024-06-03T01:08:04.285685+0000","last_peered":"2024-06-03T01:08:04.285685+0000","last_clean":"2024-06-03T01:08:04.285685+0000","last_became_active":"2024-06-03T01:05:35.253412+0000","last_became_peered":"2024-06-03T01:05:35.253412+0000","last_unstale":"2024-06-03T01:08:04.285685+0000","last_undegraded":"2024-06-03T01:08:04.285685+0000","last_fullsized":"2024-06-03T01:08:04.285685+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:12:21.890358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837806+0000","last_change":"2024-06-03T01:05:35.259007+0000","last_active":"2024-06-03T01:06:35.837806+0000","last_peered":"2024-06-03T01:06:35.837806+0000","last_clean":"2024-06-03T01:06:35.837806+0000","last_became_active":"2024-06-03T01:05:35.258358+0000","last_became_peered":"2024-06-03T01:05:35.258358+0000","last_unstale":"2024-06-03T01:06:35.837806+0000","last_undegraded":"2024-06-03T01:06:35.837806+0000","last_fullsized":"2024-06-03T01:06:35.837806+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:02.732772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"98'5","reported_seq":174,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:18.554121+0000","last_change":"2024-06-03T01:04:49.771410+0000","last_active":"2024-06-03T01:08:18.554121+0000","last_peered":"2024-06-03T01:08:18.554121+0000","last_clean":"2024-06-03T01:08:18.554121+0000","last_became_active":"2024-06-03T01:04:49.771034+0000","last_became_peered":"2024-06-03T01:04:49.771034+0000","last_unstale":"2024-06-03T01:08:18.554121+0000","last_undegraded":"2024-06-03T01:08:18.554121+0000","last_fullsized":"2024-06-03T01:08:18.554121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:25:30.888075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.266180+0000","last_change":"2024-06-03T01:04:54.363918+0000","last_active":"2024-06-03T01:08:04.266180+0000","last_peered":"2024-06-03T01:08:04.266180+0000","last_clean":"2024-06-03T01:08:04.266180+0000","last_became_active":"2024-06-03T01:04:54.363536+0000","last_became_peered":"2024-06-03T01:04:54.363536+0000","last_unstale":"2024-06-03T01:08:04.266180+0000","last_undegraded":"2024-06-03T01:08:04.266180+0000","last_fullsized":"2024-06-03T01:08:04.266180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:22.103375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.353924+0000","last_change":"2024-06-03T01:05:08.403245+0000","last_active":"2024-06-03T01:06:35.353924+0000","last_peered":"2024-06-03T01:06:35.353924+0000","last_clean":"2024-06-03T01:06:35.353924+0000","last_became_active":"2024-06-03T01:05:08.402755+0000","last_became_peered":"2024-06-03T01:05:08.402755+0000","last_unstale":"2024-06-03T01:06:35.353924+0000","last_undegraded":"2024-06-03T01:06:35.353924+0000","last_fullsized":"2024-06-03T01:06:35.353924+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:33.777084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837123+0000","last_change":"2024-06-03T01:04:49.891326+0000","last_active":"2024-06-03T01:06:35.837123+0000","last_peered":"2024-06-03T01:06:35.837123+0000","last_clean":"2024-06-03T01:06:35.837123+0000","last_became_active":"2024-06-03T01:04:49.891040+0000","last_became_peered":"2024-06-03T01:04:49.891040+0000","last_unstale":"2024-06-03T01:06:35.837123+0000","last_undegraded":"2024-06-03T01:06:35.837123+0000","last_fullsized":"2024-06-03T01:06:35.837123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:39:26.896996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.258488+0000","last_change":"2024-06-03T01:04:54.372779+0000","last_active":"2024-06-03T01:08:04.258488+0000","last_peered":"2024-06-03T01:08:04.258488+0000","last_clean":"2024-06-03T01:08:04.258488+0000","last_became_active":"2024-06-03T01:04:54.372427+0000","last_became_peered":"2024-06-03T01:04:54.372427+0000","last_unstale":"2024-06-03T01:08:04.258488+0000","last_undegraded":"2024-06-03T01:08:04.258488+0000","last_fullsized":"2024-06-03T01:08:04.258488+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:18.837300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366565+0000","last_change":"2024-06-03T01:05:08.391281+0000","last_active":"2024-06-03T01:06:35.366565+0000","last_peered":"2024-06-03T01:06:35.366565+0000","last_clean":"2024-06-03T01:06:35.366565+0000","last_became_active":"2024-06-03T01:05:08.390871+0000","last_became_peered":"2024-06-03T01:05:08.390871+0000","last_unstale":"2024-06-03T01:06:35.366565+0000","last_undegraded":"2024-06-03T01:06:35.366565+0000","last_fullsized":"2024-06-03T01:06:35.366565+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:19:12.174065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837965+0000","last_change":"2024-06-03T01:04:49.769587+0000","last_active":"2024-06-03T01:06:35.837965+0000","last_peered":"2024-06-03T01:06:35.837965+0000","last_clean":"2024-06-03T01:06:35.837965+0000","last_became_active":"2024-06-03T01:04:49.769267+0000","last_became_peered":"2024-06-03T01:04:49.769267+0000","last_unstale":"2024-06-03T01:06:35.837965+0000","last_undegraded":"2024-06-03T01:06:35.837965+0000","last_fullsized":"2024-06-03T01:06:35.837965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:18:30.747021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.254728+0000","last_change":"2024-06-03T01:04:54.368293+0000","last_active":"2024-06-03T01:08:04.254728+0000","last_peered":"2024-06-03T01:08:04.254728+0000","last_clean":"2024-06-03T01:08:04.254728+0000","last_became_active":"2024-06-03T01:04:54.367910+0000","last_became_peered":"2024-06-03T01:04:54.367910+0000","last_unstale":"2024-06-03T01:08:04.254728+0000","last_undegraded":"2024-06-03T01:08:04.254728+0000","last_fullsized":"2024-06-03T01:08:04.254728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:23:46.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837179+0000","last_change":"2024-06-03T01:05:08.619467+0000","last_active":"2024-06-03T01:06:35.837179+0000","last_peered":"2024-06-03T01:06:35.837179+0000","last_clean":"2024-06-03T01:06:35.837179+0000","last_became_active":"2024-06-03T01:05:08.618884+0000","last_became_peered":"2024-06-03T01:05:08.618884+0000","last_unstale":"2024-06-03T01:06:35.837179+0000","last_undegraded":"2024-06-03T01:06:35.837179+0000","last_fullsized":"2024-06-03T01:06:35.837179+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:22.072920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352552+0000","last_change":"2024-06-03T01:04:49.896429+0000","last_active":"2024-06-03T01:06:35.352552+0000","last_peered":"2024-06-03T01:06:35.352552+0000","last_clean":"2024-06-03T01:06:35.352552+0000","last_became_active":"2024-06-03T01:04:49.895957+0000","last_became_peered":"2024-06-03T01:04:49.895957+0000","last_unstale":"2024-06-03T01:06:35.352552+0000","last_undegraded":"2024-06-03T01:06:35.352552+0000","last_fullsized":"2024-06-03T01:06:35.352552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:52.362276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.269606+0000","last_change":"2024-06-03T01:04:54.396972+0000","last_active":"2024-06-03T01:08:04.269606+0000","last_peered":"2024-06-03T01:08:04.269606+0000","last_clean":"2024-06-03T01:08:04.269606+0000","last_became_active":"2024-06-03T01:04:54.396737+0000","last_became_peered":"2024-06-03T01:04:54.396737+0000","last_unstale":"2024-06-03T01:08:04.269606+0000","last_undegraded":"2024-06-03T01:08:04.269606+0000","last_fullsized":"2024-06-03T01:08:04.269606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:38:31.722765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351660+0000","last_change":"2024-06-03T01:05:08.390129+0000","last_active":"2024-06-03T01:06:35.351660+0000","last_peered":"2024-06-03T01:06:35.351660+0000","last_clean":"2024-06-03T01:06:35.351660+0000","last_became_active":"2024-06-03T01:05:08.389553+0000","last_became_peered":"2024-06-03T01:05:08.389553+0000","last_unstale":"2024-06-03T01:06:35.351660+0000","last_undegraded":"2024-06-03T01:06:35.351660+0000","last_fullsized":"2024-06-03T01:06:35.351660+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:53:34.176420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'7","reported_seq":175,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:17.608318+0000","last_change":"2024-06-03T01:04:49.777579+0000","last_active":"2024-06-03T01:08:17.608318+0000","last_peered":"2024-06-03T01:08:17.608318+0000","last_clean":"2024-06-03T01:08:17.608318+0000","last_became_active":"2024-06-03T01:04:49.777207+0000","last_became_peered":"2024-06-03T01:04:49.777207+0000","last_unstale":"2024-06-03T01:08:17.608318+0000","last_undegraded":"2024-06-03T01:08:17.608318+0000","last_fullsized":"2024-06-03T01:08:17.608318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:57.941162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.289556+0000","last_change":"2024-06-03T01:04:54.382883+0000","last_active":"2024-06-03T01:08:04.289556+0000","last_peered":"2024-06-03T01:08:04.289556+0000","last_clean":"2024-06-03T01:08:04.289556+0000","last_became_active":"2024-06-03T01:04:54.382445+0000","last_became_peered":"2024-06-03T01:04:54.382445+0000","last_unstale":"2024-06-03T01:08:04.289556+0000","last_undegraded":"2024-06-03T01:08:04.289556+0000","last_fullsized":"2024-06-03T01:08:04.289556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:47:07.549646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354476+0000","last_change":"2024-06-03T01:05:08.619324+0000","last_active":"2024-06-03T01:06:35.354476+0000","last_peered":"2024-06-03T01:06:35.354476+0000","last_clean":"2024-06-03T01:06:35.354476+0000","last_became_active":"2024-06-03T01:05:08.618856+0000","last_became_peered":"2024-06-03T01:05:08.618856+0000","last_unstale":"2024-06-03T01:06:35.354476+0000","last_undegraded":"2024-06-03T01:06:35.354476+0000","last_fullsized":"2024-06-03T01:06:35.354476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:55:49.520528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.260177+0000","last_change":"2024-06-03T01:04:54.371883+0000","last_active":"2024-06-03T01:08:04.260177+0000","last_peered":"2024-06-03T01:08:04.260177+0000","last_clean":"2024-06-03T01:08:04.260177+0000","last_became_active":"2024-06-03T01:04:54.371502+0000","last_became_peered":"2024-06-03T01:04:54.371502+0000","last_unstale":"2024-06-03T01:08:04.260177+0000","last_undegraded":"2024-06-03T01:08:04.260177+0000","last_fullsized":"2024-06-03T01:08:04.260177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:14.632927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:17.551360+0000","last_change":"2024-06-03T01:04:49.893131+0000","last_active":"2024-06-03T01:08:17.551360+0000","last_peered":"2024-06-03T01:08:17.551360+0000","last_clean":"2024-06-03T01:08:17.551360+0000","last_became_active":"2024-06-03T01:04:49.892122+0000","last_became_peered":"2024-06-03T01:04:49.892122+0000","last_unstale":"2024-06-03T01:08:17.551360+0000","last_undegraded":"2024-06-03T01:08:17.551360+0000","last_fullsized":"2024-06-03T01:08:17.551360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:09:07.852255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382609+0000","last_change":"2024-06-03T01:05:08.399018+0000","last_active":"2024-06-03T01:06:35.382609+0000","last_peered":"2024-06-03T01:06:35.382609+0000","last_clean":"2024-06-03T01:06:35.382609+0000","last_became_active":"2024-06-03T01:05:08.398531+0000","last_became_peered":"2024-06-03T01:05:08.398531+0000","last_unstale":"2024-06-03T01:06:35.382609+0000","last_undegraded":"2024-06-03T01:06:35.382609+0000","last_fullsized":"2024-06-03T01:06:35.382609+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:54.115344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.267539+0000","last_change":"2024-06-03T01:04:54.385215+0000","last_active":"2024-06-03T01:08:04.267539+0000","last_peered":"2024-06-03T01:08:04.267539+0000","last_clean":"2024-06-03T01:08:04.267539+0000","last_became_active":"2024-06-03T01:04:54.384764+0000","last_became_peered":"2024-06-03T01:04:54.384764+0000","last_unstale":"2024-06-03T01:08:04.267539+0000","last_undegraded":"2024-06-03T01:08:04.267539+0000","last_fullsized":"2024-06-03T01:08:04.267539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:45.388604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354880+0000","last_change":"2024-06-03T01:04:49.781444+0000","last_active":"2024-06-03T01:06:35.354880+0000","last_peered":"2024-06-03T01:06:35.354880+0000","last_clean":"2024-06-03T01:06:35.354880+0000","last_became_active":"2024-06-03T01:04:49.780922+0000","last_became_peered":"2024-06-03T01:04:49.780922+0000","last_unstale":"2024-06-03T01:06:35.354880+0000","last_undegraded":"2024-06-03T01:06:35.354880+0000","last_fullsized":"2024-06-03T01:06:35.354880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:19.843289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":379,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382057+0000","last_change":"2024-06-03T01:03:16.011282+0000","last_active":"2024-06-03T01:06:35.382057+0000","last_peered":"2024-06-03T01:06:35.382057+0000","last_clean":"2024-06-03T01:06:35.382057+0000","last_became_active":"2024-06-03T01:03:16.010726+0000","last_became_peered":"2024-06-03T01:03:16.010726+0000","last_unstale":"2024-06-03T01:06:35.382057+0000","last_undegraded":"2024-06-03T01:06:35.382057+0000","last_fullsized":"2024-06-03T01:06:35.382057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_clean_scrub_stamp":"2024-06-03T00:59:20.780015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:50:32.649940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.367062+0000","last_change":"2024-06-03T01:05:08.620410+0000","last_active":"2024-06-03T01:06:35.367062+0000","last_peered":"2024-06-03T01:06:35.367062+0000","last_clean":"2024-06-03T01:06:35.367062+0000","last_became_active":"2024-06-03T01:05:08.619601+0000","last_became_peered":"2024-06-03T01:05:08.619601+0000","last_unstale":"2024-06-03T01:06:35.367062+0000","last_undegraded":"2024-06-03T01:06:35.367062+0000","last_fullsized":"2024-06-03T01:06:35.367062+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:47.073077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.252417+0000","last_change":"2024-06-03T01:04:54.379373+0000","last_active":"2024-06-03T01:08:04.252417+0000","last_peered":"2024-06-03T01:08:04.252417+0000","last_clean":"2024-06-03T01:08:04.252417+0000","last_became_active":"2024-06-03T01:04:54.378981+0000","last_became_peered":"2024-06-03T01:04:54.378981+0000","last_unstale":"2024-06-03T01:08:04.252417+0000","last_undegraded":"2024-06-03T01:08:04.252417+0000","last_fullsized":"2024-06-03T01:08:04.252417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:27:06.651091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382211+0000","last_change":"2024-06-03T01:04:49.773896+0000","last_active":"2024-06-03T01:06:35.382211+0000","last_peered":"2024-06-03T01:06:35.382211+0000","last_clean":"2024-06-03T01:06:35.382211+0000","last_became_active":"2024-06-03T01:04:49.773403+0000","last_became_peered":"2024-06-03T01:04:49.773403+0000","last_unstale":"2024-06-03T01:06:35.382211+0000","last_undegraded":"2024-06-03T01:06:35.382211+0000","last_fullsized":"2024-06-03T01:06:35.382211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:34:13.376589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354565+0000","last_change":"2024-06-03T01:05:08.389575+0000","last_active":"2024-06-03T01:06:35.354565+0000","last_peered":"2024-06-03T01:06:35.354565+0000","last_clean":"2024-06-03T01:06:35.354565+0000","last_became_active":"2024-06-03T01:05:08.389010+0000","last_became_peered":"2024-06-03T01:05:08.389010+0000","last_unstale":"2024-06-03T01:06:35.354565+0000","last_undegraded":"2024-06-03T01:06:35.354565+0000","last_fullsized":"2024-06-03T01:06:35.354565+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:57:26.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354714+0000","last_change":"2024-06-03T01:05:35.263346+0000","last_active":"2024-06-03T01:06:35.354714+0000","last_peered":"2024-06-03T01:06:35.354714+0000","last_clean":"2024-06-03T01:06:35.354714+0000","last_became_active":"2024-06-03T01:05:35.262843+0000","last_became_peered":"2024-06-03T01:05:35.262843+0000","last_unstale":"2024-06-03T01:06:35.354714+0000","last_undegraded":"2024-06-03T01:06:35.354714+0000","last_fullsized":"2024-06-03T01:06:35.354714+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:44.247086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.283246+0000","last_change":"2024-06-03T01:04:54.388176+0000","last_active":"2024-06-03T01:08:04.283246+0000","last_peered":"2024-06-03T01:08:04.283246+0000","last_clean":"2024-06-03T01:08:04.283246+0000","last_became_active":"2024-06-03T01:04:54.384544+0000","last_became_peered":"2024-06-03T01:04:54.384544+0000","last_unstale":"2024-06-03T01:08:04.283246+0000","last_undegraded":"2024-06-03T01:08:04.283246+0000","last_fullsized":"2024-06-03T01:08:04.283246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:00:42.931031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839686+0000","last_change":"2024-06-03T01:05:08.619540+0000","last_active":"2024-06-03T01:06:35.839686+0000","last_peered":"2024-06-03T01:06:35.839686+0000","last_clean":"2024-06-03T01:06:35.839686+0000","last_became_active":"2024-06-03T01:05:08.618834+0000","last_became_peered":"2024-06-03T01:05:08.618834+0000","last_unstale":"2024-06-03T01:06:35.839686+0000","last_undegraded":"2024-06-03T01:06:35.839686+0000","last_fullsized":"2024-06-03T01:06:35.839686+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:07:05.467161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'17","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:07:00.330523+0000","last_change":"2024-06-03T01:04:49.786635+0000","last_active":"2024-06-03T01:07:00.330523+0000","last_peered":"2024-06-03T01:07:00.330523+0000","last_clean":"2024-06-03T01:07:00.330523+0000","last_became_active":"2024-06-03T01:04:49.785951+0000","last_became_peered":"2024-06-03T01:04:49.785951+0000","last_unstale":"2024-06-03T01:07:00.330523+0000","last_undegraded":"2024-06-03T01:07:00.330523+0000","last_fullsized":"2024-06-03T01:07:00.330523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:10.784322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.273463+0000","last_change":"2024-06-03T01:04:54.387111+0000","last_active":"2024-06-03T01:08:04.273463+0000","last_peered":"2024-06-03T01:08:04.273463+0000","last_clean":"2024-06-03T01:08:04.273463+0000","last_became_active":"2024-06-03T01:04:54.384691+0000","last_became_peered":"2024-06-03T01:04:54.384691+0000","last_unstale":"2024-06-03T01:08:04.273463+0000","last_undegraded":"2024-06-03T01:08:04.273463+0000","last_fullsized":"2024-06-03T01:08:04.273463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:11:15.254855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347498+0000","last_change":"2024-06-03T01:05:08.386335+0000","last_active":"2024-06-03T01:06:35.347498+0000","last_peered":"2024-06-03T01:06:35.347498+0000","last_clean":"2024-06-03T01:06:35.347498+0000","last_became_active":"2024-06-03T01:05:08.385722+0000","last_became_peered":"2024-06-03T01:05:08.385722+0000","last_unstale":"2024-06-03T01:06:35.347498+0000","last_undegraded":"2024-06-03T01:06:35.347498+0000","last_fullsized":"2024-06-03T01:06:35.347498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:00.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355343+0000","last_change":"2024-06-03T01:04:49.883125+0000","last_active":"2024-06-03T01:06:35.355343+0000","last_peered":"2024-06-03T01:06:35.355343+0000","last_clean":"2024-06-03T01:06:35.355343+0000","last_became_active":"2024-06-03T01:04:49.882760+0000","last_became_peered":"2024-06-03T01:04:49.882760+0000","last_unstale":"2024-06-03T01:06:35.355343+0000","last_undegraded":"2024-06-03T01:06:35.355343+0000","last_fullsized":"2024-06-03T01:06:35.355343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:05:30.854654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290667+0000","last_change":"2024-06-03T01:04:54.386265+0000","last_active":"2024-06-03T01:08:04.290667+0000","last_peered":"2024-06-03T01:08:04.290667+0000","last_clean":"2024-06-03T01:08:04.290667+0000","last_became_active":"2024-06-03T01:04:54.385801+0000","last_became_peered":"2024-06-03T01:04:54.385801+0000","last_unstale":"2024-06-03T01:08:04.290667+0000","last_undegraded":"2024-06-03T01:08:04.290667+0000","last_fullsized":"2024-06-03T01:08:04.290667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:40:26.128516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355206+0000","last_change":"2024-06-03T01:05:08.619166+0000","last_active":"2024-06-03T01:06:35.355206+0000","last_peered":"2024-06-03T01:06:35.355206+0000","last_clean":"2024-06-03T01:06:35.355206+0000","last_became_active":"2024-06-03T01:05:08.618601+0000","last_became_peered":"2024-06-03T01:05:08.618601+0000","last_unstale":"2024-06-03T01:06:35.355206+0000","last_undegraded":"2024-06-03T01:06:35.355206+0000","last_fullsized":"2024-06-03T01:06:35.355206+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:32.401683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837348+0000","last_change":"2024-06-03T01:05:35.258905+0000","last_active":"2024-06-03T01:06:35.837348+0000","last_peered":"2024-06-03T01:06:35.837348+0000","last_clean":"2024-06-03T01:06:35.837348+0000","last_became_active":"2024-06-03T01:05:35.258423+0000","last_became_peered":"2024-06-03T01:05:35.258423+0000","last_unstale":"2024-06-03T01:06:35.837348+0000","last_undegraded":"2024-06-03T01:06:35.837348+0000","last_fullsized":"2024-06-03T01:06:35.837348+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:31:36.969581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.222910+0000","last_change":"2024-06-03T01:04:54.367788+0000","last_active":"2024-06-03T01:08:04.222910+0000","last_peered":"2024-06-03T01:08:04.222910+0000","last_clean":"2024-06-03T01:08:04.222910+0000","last_became_active":"2024-06-03T01:04:54.367353+0000","last_became_peered":"2024-06-03T01:04:54.367353+0000","last_unstale":"2024-06-03T01:08:04.222910+0000","last_undegraded":"2024-06-03T01:08:04.222910+0000","last_fullsized":"2024-06-03T01:08:04.222910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:26.604176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352869+0000","last_change":"2024-06-03T01:05:08.405875+0000","last_active":"2024-06-03T01:06:35.352869+0000","last_peered":"2024-06-03T01:06:35.352869+0000","last_clean":"2024-06-03T01:06:35.352869+0000","last_became_active":"2024-06-03T01:05:08.405410+0000","last_became_peered":"2024-06-03T01:05:08.405410+0000","last_unstale":"2024-06-03T01:06:35.352869+0000","last_undegraded":"2024-06-03T01:06:35.352869+0000","last_fullsized":"2024-06-03T01:06:35.352869+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:19:51.467150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840215+0000","last_change":"2024-06-03T01:04:49.786446+0000","last_active":"2024-06-03T01:06:35.840215+0000","last_peered":"2024-06-03T01:06:35.840215+0000","last_clean":"2024-06-03T01:06:35.840215+0000","last_became_active":"2024-06-03T01:04:49.786036+0000","last_became_peered":"2024-06-03T01:04:49.786036+0000","last_unstale":"2024-06-03T01:06:35.840215+0000","last_undegraded":"2024-06-03T01:06:35.840215+0000","last_fullsized":"2024-06-03T01:06:35.840215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:41:12.176583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288101+0000","last_change":"2024-06-03T01:04:54.382742+0000","last_active":"2024-06-03T01:08:04.288101+0000","last_peered":"2024-06-03T01:08:04.288101+0000","last_clean":"2024-06-03T01:08:04.288101+0000","last_became_active":"2024-06-03T01:04:54.382296+0000","last_became_peered":"2024-06-03T01:04:54.382296+0000","last_unstale":"2024-06-03T01:08:04.288101+0000","last_undegraded":"2024-06-03T01:08:04.288101+0000","last_fullsized":"2024-06-03T01:08:04.288101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:32:03.385387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"100'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:39.228380+0000","last_change":"2024-06-03T01:05:08.403233+0000","last_active":"2024-06-03T01:06:39.228380+0000","last_peered":"2024-06-03T01:06:39.228380+0000","last_clean":"2024-06-03T01:06:39.228380+0000","last_became_active":"2024-06-03T01:05:08.402693+0000","last_became_peered":"2024-06-03T01:05:08.402693+0000","last_unstale":"2024-06-03T01:06:39.228380+0000","last_undegraded":"2024-06-03T01:06:39.228380+0000","last_fullsized":"2024-06-03T01:06:39.228380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:35:30.722557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347626+0000","last_change":"2024-06-03T01:04:49.779089+0000","last_active":"2024-06-03T01:06:35.347626+0000","last_peered":"2024-06-03T01:06:35.347626+0000","last_clean":"2024-06-03T01:06:35.347626+0000","last_became_active":"2024-06-03T01:04:49.778162+0000","last_became_peered":"2024-06-03T01:04:49.778162+0000","last_unstale":"2024-06-03T01:06:35.347626+0000","last_undegraded":"2024-06-03T01:06:35.347626+0000","last_fullsized":"2024-06-03T01:06:35.347626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:43.151731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.291790+0000","last_change":"2024-06-03T01:04:54.393447+0000","last_active":"2024-06-03T01:08:04.291790+0000","last_peered":"2024-06-03T01:08:04.291790+0000","last_clean":"2024-06-03T01:08:04.291790+0000","last_became_active":"2024-06-03T01:04:54.390154+0000","last_became_peered":"2024-06-03T01:04:54.390154+0000","last_unstale":"2024-06-03T01:08:04.291790+0000","last_undegraded":"2024-06-03T01:08:04.291790+0000","last_fullsized":"2024-06-03T01:08:04.291790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:00:21.854391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355124+0000","last_change":"2024-06-03T01:05:08.619603+0000","last_active":"2024-06-03T01:06:35.355124+0000","last_peered":"2024-06-03T01:06:35.355124+0000","last_clean":"2024-06-03T01:06:35.355124+0000","last_became_active":"2024-06-03T01:05:08.618554+0000","last_became_peered":"2024-06-03T01:05:08.618554+0000","last_unstale":"2024-06-03T01:06:35.355124+0000","last_undegraded":"2024-06-03T01:06:35.355124+0000","last_fullsized":"2024-06-03T01:06:35.355124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:57:26.125390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837454+0000","last_change":"2024-06-03T01:04:49.770531+0000","last_active":"2024-06-03T01:06:35.837454+0000","last_peered":"2024-06-03T01:06:35.837454+0000","last_clean":"2024-06-03T01:06:35.837454+0000","last_became_active":"2024-06-03T01:04:49.770217+0000","last_became_peered":"2024-06-03T01:04:49.770217+0000","last_unstale":"2024-06-03T01:06:35.837454+0000","last_undegraded":"2024-06-03T01:06:35.837454+0000","last_fullsized":"2024-06-03T01:06:35.837454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:59:52.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.277547+0000","last_change":"2024-06-03T01:04:54.376511+0000","last_active":"2024-06-03T01:08:04.277547+0000","last_peered":"2024-06-03T01:08:04.277547+0000","last_clean":"2024-06-03T01:08:04.277547+0000","last_became_active":"2024-06-03T01:04:54.375896+0000","last_became_peered":"2024-06-03T01:04:54.375896+0000","last_unstale":"2024-06-03T01:08:04.277547+0000","last_undegraded":"2024-06-03T01:08:04.277547+0000","last_fullsized":"2024-06-03T01:08:04.277547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:20.237450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837429+0000","last_change":"2024-06-03T01:05:08.387551+0000","last_active":"2024-06-03T01:06:35.837429+0000","last_peered":"2024-06-03T01:06:35.837429+0000","last_clean":"2024-06-03T01:06:35.837429+0000","last_became_active":"2024-06-03T01:05:08.386719+0000","last_became_peered":"2024-06-03T01:05:08.386719+0000","last_unstale":"2024-06-03T01:06:35.837429+0000","last_undegraded":"2024-06-03T01:06:35.837429+0000","last_fullsized":"2024-06-03T01:06:35.837429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:03:57.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":178,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:18.598962+0000","last_change":"2024-06-03T01:04:49.890435+0000","last_active":"2024-06-03T01:08:18.598962+0000","last_peered":"2024-06-03T01:08:18.598962+0000","last_clean":"2024-06-03T01:08:18.598962+0000","last_became_active":"2024-06-03T01:04:49.890074+0000","last_became_peered":"2024-06-03T01:04:49.890074+0000","last_unstale":"2024-06-03T01:08:18.598962+0000","last_undegraded":"2024-06-03T01:08:18.598962+0000","last_fullsized":"2024-06-03T01:08:18.598962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:48:46.743934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.268967+0000","last_change":"2024-06-03T01:04:54.366487+0000","last_active":"2024-06-03T01:08:04.268967+0000","last_peered":"2024-06-03T01:08:04.268967+0000","last_clean":"2024-06-03T01:08:04.268967+0000","last_became_active":"2024-06-03T01:04:54.365905+0000","last_became_peered":"2024-06-03T01:04:54.365905+0000","last_unstale":"2024-06-03T01:08:04.268967+0000","last_undegraded":"2024-06-03T01:08:04.268967+0000","last_fullsized":"2024-06-03T01:08:04.268967+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:04:09.159028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597029+0000","last_change":"2024-06-03T01:05:08.404608+0000","last_active":"2024-06-03T01:06:35.597029+0000","last_peered":"2024-06-03T01:06:35.597029+0000","last_clean":"2024-06-03T01:06:35.597029+0000","last_became_active":"2024-06-03T01:05:08.404148+0000","last_became_peered":"2024-06-03T01:05:08.404148+0000","last_unstale":"2024-06-03T01:06:35.597029+0000","last_undegraded":"2024-06-03T01:06:35.597029+0000","last_fullsized":"2024-06-03T01:06:35.597029+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:29:54.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366988+0000","last_change":"2024-06-03T01:04:49.778805+0000","last_active":"2024-06-03T01:06:35.366988+0000","last_peered":"2024-06-03T01:06:35.366988+0000","last_clean":"2024-06-03T01:06:35.366988+0000","last_became_active":"2024-06-03T01:04:49.778436+0000","last_became_peered":"2024-06-03T01:04:49.778436+0000","last_unstale":"2024-06-03T01:06:35.366988+0000","last_undegraded":"2024-06-03T01:06:35.366988+0000","last_fullsized":"2024-06-03T01:06:35.366988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:21.369521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.271058+0000","last_change":"2024-06-03T01:04:54.393604+0000","last_active":"2024-06-03T01:08:04.271058+0000","last_peered":"2024-06-03T01:08:04.271058+0000","last_clean":"2024-06-03T01:08:04.271058+0000","last_became_active":"2024-06-03T01:04:54.393282+0000","last_became_peered":"2024-06-03T01:04:54.393282+0000","last_unstale":"2024-06-03T01:08:04.271058+0000","last_undegraded":"2024-06-03T01:08:04.271058+0000","last_fullsized":"2024-06-03T01:08:04.271058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:17:24.435502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366426+0000","last_change":"2024-06-03T01:05:08.386399+0000","last_active":"2024-06-03T01:06:35.366426+0000","last_peered":"2024-06-03T01:06:35.366426+0000","last_clean":"2024-06-03T01:06:35.366426+0000","last_became_active":"2024-06-03T01:05:08.384547+0000","last_became_peered":"2024-06-03T01:05:08.384547+0000","last_unstale":"2024-06-03T01:06:35.366426+0000","last_undegraded":"2024-06-03T01:06:35.366426+0000","last_fullsized":"2024-06-03T01:06:35.366426+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:04:17.097695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354216+0000","last_change":"2024-06-03T01:04:49.782877+0000","last_active":"2024-06-03T01:06:35.354216+0000","last_peered":"2024-06-03T01:06:35.354216+0000","last_clean":"2024-06-03T01:06:35.354216+0000","last_became_active":"2024-06-03T01:04:49.781399+0000","last_became_peered":"2024-06-03T01:04:49.781399+0000","last_unstale":"2024-06-03T01:06:35.354216+0000","last_undegraded":"2024-06-03T01:06:35.354216+0000","last_fullsized":"2024-06-03T01:06:35.354216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:16.150699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.266677+0000","last_change":"2024-06-03T01:04:54.365914+0000","last_active":"2024-06-03T01:08:04.266677+0000","last_peered":"2024-06-03T01:08:04.266677+0000","last_clean":"2024-06-03T01:08:04.266677+0000","last_became_active":"2024-06-03T01:04:54.365507+0000","last_became_peered":"2024-06-03T01:04:54.365507+0000","last_unstale":"2024-06-03T01:08:04.266677+0000","last_undegraded":"2024-06-03T01:08:04.266677+0000","last_fullsized":"2024-06-03T01:08:04.266677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:24:04.278102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.260307+0000","last_change":"2024-06-03T01:04:54.367546+0000","last_active":"2024-06-03T01:08:04.260307+0000","last_peered":"2024-06-03T01:08:04.260307+0000","last_clean":"2024-06-03T01:08:04.260307+0000","last_became_active":"2024-06-03T01:04:54.367116+0000","last_became_peered":"2024-06-03T01:04:54.367116+0000","last_unstale":"2024-06-03T01:08:04.260307+0000","last_undegraded":"2024-06-03T01:08:04.260307+0000","last_fullsized":"2024-06-03T01:08:04.260307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:52:15.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.255857+0000","last_change":"2024-06-03T01:04:54.366779+0000","last_active":"2024-06-03T01:08:04.255857+0000","last_peered":"2024-06-03T01:08:04.255857+0000","last_clean":"2024-06-03T01:08:04.255857+0000","last_became_active":"2024-06-03T01:04:54.366420+0000","last_became_peered":"2024-06-03T01:04:54.366420+0000","last_unstale":"2024-06-03T01:08:04.255857+0000","last_undegraded":"2024-06-03T01:08:04.255857+0000","last_fullsized":"2024-06-03T01:08:04.255857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:31:16.277064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.261192+0000","last_change":"2024-06-03T01:04:54.393396+0000","last_active":"2024-06-03T01:08:04.261192+0000","last_peered":"2024-06-03T01:08:04.261192+0000","last_clean":"2024-06-03T01:08:04.261192+0000","last_became_active":"2024-06-03T01:04:54.393071+0000","last_became_peered":"2024-06-03T01:04:54.393071+0000","last_unstale":"2024-06-03T01:08:04.261192+0000","last_undegraded":"2024-06-03T01:08:04.261192+0000","last_fullsized":"2024-06-03T01:08:04.261192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:25.507494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290933+0000","last_change":"2024-06-03T01:05:35.262012+0000","last_active":"2024-06-03T01:08:04.290933+0000","last_peered":"2024-06-03T01:08:04.290933+0000","last_clean":"2024-06-03T01:08:04.290933+0000","last_became_active":"2024-06-03T01:05:35.261644+0000","last_became_peered":"2024-06-03T01:05:35.261644+0000","last_unstale":"2024-06-03T01:08:04.290933+0000","last_undegraded":"2024-06-03T01:08:04.290933+0000","last_fullsized":"2024-06-03T01:08:04.290933+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:33.480127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288883+0000","last_change":"2024-06-03T01:04:54.388884+0000","last_active":"2024-06-03T01:08:04.288883+0000","last_peered":"2024-06-03T01:08:04.288883+0000","last_clean":"2024-06-03T01:08:04.288883+0000","last_became_active":"2024-06-03T01:04:54.387969+0000","last_became_peered":"2024-06-03T01:04:54.387969+0000","last_unstale":"2024-06-03T01:08:04.288883+0000","last_undegraded":"2024-06-03T01:08:04.288883+0000","last_fullsized":"2024-06-03T01:08:04.288883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:13.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.254966+0000","last_change":"2024-06-03T01:04:54.364412+0000","last_active":"2024-06-03T01:08:04.254966+0000","last_peered":"2024-06-03T01:08:04.254966+0000","last_clean":"2024-06-03T01:08:04.254966+0000","last_became_active":"2024-06-03T01:04:54.364053+0000","last_became_peered":"2024-06-03T01:04:54.364053+0000","last_unstale":"2024-06-03T01:08:04.254966+0000","last_undegraded":"2024-06-03T01:08:04.254966+0000","last_fullsized":"2024-06-03T01:08:04.254966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:26.760514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.257347+0000","last_change":"2024-06-03T01:04:54.370604+0000","last_active":"2024-06-03T01:08:04.257347+0000","last_peered":"2024-06-03T01:08:04.257347+0000","last_clean":"2024-06-03T01:08:04.257347+0000","last_became_active":"2024-06-03T01:04:54.370288+0000","last_became_peered":"2024-06-03T01:04:54.370288+0000","last_unstale":"2024-06-03T01:08:04.257347+0000","last_undegraded":"2024-06-03T01:08:04.257347+0000","last_fullsized":"2024-06-03T01:08:04.257347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:34:49.350481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.274458+0000","last_change":"2024-06-03T01:05:35.260161+0000","last_active":"2024-06-03T01:08:04.274458+0000","last_peered":"2024-06-03T01:08:04.274458+0000","last_clean":"2024-06-03T01:08:04.274458+0000","last_became_active":"2024-06-03T01:05:35.259725+0000","last_became_peered":"2024-06-03T01:05:35.259725+0000","last_unstale":"2024-06-03T01:08:04.274458+0000","last_undegraded":"2024-06-03T01:08:04.274458+0000","last_fullsized":"2024-06-03T01:08:04.274458+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:36.698726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.275477+0000","last_change":"2024-06-03T01:04:54.391161+0000","last_active":"2024-06-03T01:08:04.275477+0000","last_peered":"2024-06-03T01:08:04.275477+0000","last_clean":"2024-06-03T01:08:04.275477+0000","last_became_active":"2024-06-03T01:04:54.390789+0000","last_became_peered":"2024-06-03T01:04:54.390789+0000","last_unstale":"2024-06-03T01:08:04.275477+0000","last_undegraded":"2024-06-03T01:08:04.275477+0000","last_fullsized":"2024-06-03T01:08:04.275477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:54:07.520300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.280657+0000","last_change":"2024-06-03T01:05:35.257377+0000","last_active":"2024-06-03T01:08:04.280657+0000","last_peered":"2024-06-03T01:08:04.280657+0000","last_clean":"2024-06-03T01:08:04.280657+0000","last_became_active":"2024-06-03T01:05:35.256864+0000","last_became_peered":"2024-06-03T01:05:35.256864+0000","last_unstale":"2024-06-03T01:08:04.280657+0000","last_undegraded":"2024-06-03T01:08:04.280657+0000","last_fullsized":"2024-06-03T01:08:04.280657+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:06:16.396763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.271939+0000","last_change":"2024-06-03T01:04:54.385235+0000","last_active":"2024-06-03T01:08:04.271939+0000","last_peered":"2024-06-03T01:08:04.271939+0000","last_clean":"2024-06-03T01:08:04.271939+0000","last_became_active":"2024-06-03T01:04:54.384834+0000","last_became_peered":"2024-06-03T01:04:54.384834+0000","last_unstale":"2024-06-03T01:08:04.271939+0000","last_undegraded":"2024-06-03T01:08:04.271939+0000","last_fullsized":"2024-06-03T01:08:04.271939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:49.586108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.292093+0000","last_change":"2024-06-03T01:04:54.389627+0000","last_active":"2024-06-03T01:08:04.292093+0000","last_peered":"2024-06-03T01:08:04.292093+0000","last_clean":"2024-06-03T01:08:04.292093+0000","last_became_active":"2024-06-03T01:04:54.388430+0000","last_became_peered":"2024-06-03T01:04:54.388430+0000","last_unstale":"2024-06-03T01:08:04.292093+0000","last_undegraded":"2024-06-03T01:08:04.292093+0000","last_fullsized":"2024-06-03T01:08:04.292093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:32:44.376952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.254013+0000","last_change":"2024-06-03T01:04:54.366394+0000","last_active":"2024-06-03T01:08:04.254013+0000","last_peered":"2024-06-03T01:08:04.254013+0000","last_clean":"2024-06-03T01:08:04.254013+0000","last_became_active":"2024-06-03T01:04:54.366067+0000","last_became_peered":"2024-06-03T01:04:54.366067+0000","last_unstale":"2024-06-03T01:08:04.254013+0000","last_undegraded":"2024-06-03T01:08:04.254013+0000","last_fullsized":"2024-06-03T01:08:04.254013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:18:03.772654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76846,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1272,"num_read_kb":724,"num_write":266,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":230538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351035,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":575494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612673,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37152,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708000,"statfs":{"total":95995035648,"available":95956992000,"internally_reserved":0,"allocated":1859584,"data_stored":1230498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776199,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":578877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037837,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":575047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299474,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1777664,"data_stored":1172327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463000,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32308,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712844,"statfs":{"total":95995035648,"available":95961952256,"internally_reserved":0,"allocated":1159168,"data_stored":570816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1232896,"data_stored":640677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":572272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1253376,"data_stored":641428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":569993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920867,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":574359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1785856,"data_stored":1167378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T01:08:24.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-03T01:08:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:24 smithi107 ceph-mon[36622]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:08:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:24 smithi107 ceph-mon[36622]: pgmap v445: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:24 smithi179 ceph-mon[37041]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:08:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:24 smithi179 ceph-mon[37041]: pgmap v445: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:25.184 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:26 smithi094 ceph-mon[32119]: pgmap v446: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:26 smithi107 ceph-mon[36622]: pgmap v446: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:26 smithi179 ceph-mon[37041]: pgmap v446: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.528+0000 7f78da000700 1 -- 172.21.15.94:0/2083278625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d41089d0 msgr2=0x7f78d4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.528+0000 7f78da000700 1 --2- 172.21.15.94:0/2083278625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d41089d0 0x7f78d4108e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009990 tx=0x7f78c0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.528+0000 7f78da000700 1 -- 172.21.15.94:0/2083278625 shutdown_connections 2024-06-03T01:08:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.528+0000 7f78da000700 1 --2- 172.21.15.94:0/2083278625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78d4109390 0x7f78d4115850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.528+0000 7f78da000700 1 --2- 172.21.15.94:0/2083278625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d410e980 0x7f78d410ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.528+0000 7f78da000700 1 --2- 172.21.15.94:0/2083278625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d41089d0 0x7f78d4108e50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.528+0000 7f78da000700 1 -- 172.21.15.94:0/2083278625 >> 172.21.15.94:0/2083278625 conn(0x7f78d4078200 msgr2=0x7f78d41072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.529+0000 7f78da000700 1 -- 172.21.15.94:0/2083278625 shutdown_connections 2024-06-03T01:08:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.529+0000 7f78da000700 1 -- 172.21.15.94:0/2083278625 wait complete. 2024-06-03T01:08:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 Processor -- start 2024-06-03T01:08:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 -- start start 2024-06-03T01:08:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d41089d0 0x7f78d419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4109390 0x7f78d419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78d410e980 0x7f78d41a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78d4118ce0 con 0x7f78d41089d0 2024-06-03T01:08:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78d4118b60 con 0x7f78d4109390 2024-06-03T01:08:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.530+0000 7f78da000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78d4118e60 con 0x7f78d410e980 2024-06-03T01:08:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4109390 0x7f78d419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d8c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d41089d0 0x7f78d419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4109390 0x7f78d419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:53426/0 (socket says 172.21.15.94:53426) 2024-06-03T01:08:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d9600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78d410e980 0x7f78d41a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 -- 172.21.15.94:0/234285281 learned_addr learned my addr 172.21.15.94:0/234285281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 -- 172.21.15.94:0/234285281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78d410e980 msgr2=0x7f78d41a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:27.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78d410e980 0x7f78d41a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 -- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d41089d0 msgr2=0x7f78d419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d41089d0 0x7f78d419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.531+0000 7f78d3e00700 1 -- 172.21.15.94:0/234285281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d41a40d0 con 0x7f78d4109390 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.532+0000 7f78d3e00700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4109390 0x7f78d419f650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78c000bae0 tx=0x7f78c000bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.532+0000 7f78d1600700 1 -- 172.21.15.94:0/234285281 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0026070 con 0x7f78d4109390 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.532+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d41a4360 con 0x7f78d4109390 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.532+0000 7f78d1600700 1 -- 172.21.15.94:0/234285281 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c0003d70 con 0x7f78d4109390 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.532+0000 7f78d1600700 1 -- 172.21.15.94:0/234285281 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0021410 con 0x7f78d4109390 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.532+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d41abc40 con 0x7f78d4109390 2024-06-03T01:08:27.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.533+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d4069180 con 0x7f78d4109390 2024-06-03T01:08:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.535+0000 7f78d1600700 1 -- 172.21.15.94:0/234285281 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f78c0029070 con 0x7f78d4109390 2024-06-03T01:08:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.537+0000 7f78d1600700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f78bc0a2ca0 0x7f78bc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.537+0000 7f78d1600700 1 -- 172.21.15.94:0/234285281 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f78c00d0610 con 0x7f78d4109390 2024-06-03T01:08:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.537+0000 7f78d8c00700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f78bc0a2ca0 0x7f78bc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.538+0000 7f78d8c00700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f78bc0a2ca0 0x7f78bc0a5160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009be0 tx=0x7f78c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:27.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.539+0000 7f78d1600700 1 -- 172.21.15.94:0/234285281 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f78c0030080 con 0x7f78d4109390 2024-06-03T01:08:27.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.758+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78d41a0420 con 0x7f78bc0a2ca0 2024-06-03T01:08:27.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.775+0000 7f78d1600700 1 -- 172.21.15.94:0/234285281 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340725 (secure 0 0 0) 0x7f78d41a0420 con 0x7f78bc0a2ca0 2024-06-03T01:08:27.781 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:08:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.781+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f78bc0a2ca0 msgr2=0x7f78bc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f78bc0a2ca0 0x7f78bc0a5160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009be0 tx=0x7f78c8009260 comp rx=0 tx=0).stop 2024-06-03T01:08:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4109390 msgr2=0x7f78d419f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4109390 0x7f78d419f650 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78c000bae0 tx=0x7f78c000bb10 comp rx=0 tx=0).stop 2024-06-03T01:08:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 shutdown_connections 2024-06-03T01:08:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f78bc0a2ca0 0x7f78bc0a5160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78d410e980 0x7f78d41a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78d41089d0 0x7f78d419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 --2- 172.21.15.94:0/234285281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d4109390 0x7f78d419f650 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 >> 172.21.15.94:0/234285281 conn(0x7f78d4078200 msgr2=0x7f78d4078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 shutdown_connections 2024-06-03T01:08:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:27.782+0000 7f78da000700 1 -- 172.21.15.94:0/234285281 wait complete. 2024-06-03T01:08:27.787 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-06-03T01:08:28.671 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-06-03T01:08:26.353692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667448,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1437,"num_read_kb":853,"num_write":449,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":410652,"kb_used_data":15740,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124531172,"statfs":{"total":1151940427776,"available":1151519920128,"internally_reserved":0,"allocated":16117760,"data_stored":8869166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":404272442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005819"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288498+0000","last_change":"2024-06-03T01:04:54.382830+0000","last_active":"2024-06-03T01:08:04.288498+0000","last_peered":"2024-06-03T01:08:04.288498+0000","last_clean":"2024-06-03T01:08:04.288498+0000","last_became_active":"2024-06-03T01:04:54.382328+0000","last_became_peered":"2024-06-03T01:04:54.382328+0000","last_unstale":"2024-06-03T01:08:04.288498+0000","last_undegraded":"2024-06-03T01:08:04.288498+0000","last_fullsized":"2024-06-03T01:08:04.288498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:59.037188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288252+0000","last_change":"2024-06-03T01:04:54.380179+0000","last_active":"2024-06-03T01:08:04.288252+0000","last_peered":"2024-06-03T01:08:04.288252+0000","last_clean":"2024-06-03T01:08:04.288252+0000","last_became_active":"2024-06-03T01:04:54.370975+0000","last_became_peered":"2024-06-03T01:04:54.370975+0000","last_unstale":"2024-06-03T01:08:04.288252+0000","last_undegraded":"2024-06-03T01:08:04.288252+0000","last_fullsized":"2024-06-03T01:08:04.288252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:14:41.133053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290162+0000","last_change":"2024-06-03T01:04:54.381423+0000","last_active":"2024-06-03T01:08:04.290162+0000","last_peered":"2024-06-03T01:08:04.290162+0000","last_clean":"2024-06-03T01:08:04.290162+0000","last_became_active":"2024-06-03T01:04:54.375659+0000","last_became_peered":"2024-06-03T01:04:54.375659+0000","last_unstale":"2024-06-03T01:08:04.290162+0000","last_undegraded":"2024-06-03T01:08:04.290162+0000","last_fullsized":"2024-06-03T01:08:04.290162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:50.294425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.258917+0000","last_change":"2024-06-03T01:04:54.372494+0000","last_active":"2024-06-03T01:08:04.258917+0000","last_peered":"2024-06-03T01:08:04.258917+0000","last_clean":"2024-06-03T01:08:04.258917+0000","last_became_active":"2024-06-03T01:04:54.371744+0000","last_became_peered":"2024-06-03T01:04:54.371744+0000","last_unstale":"2024-06-03T01:08:04.258917+0000","last_undegraded":"2024-06-03T01:08:04.258917+0000","last_fullsized":"2024-06-03T01:08:04.258917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:28:01.887751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.272884+0000","last_change":"2024-06-03T01:04:54.394675+0000","last_active":"2024-06-03T01:08:04.272884+0000","last_peered":"2024-06-03T01:08:04.272884+0000","last_clean":"2024-06-03T01:08:04.272884+0000","last_became_active":"2024-06-03T01:04:54.394246+0000","last_became_peered":"2024-06-03T01:04:54.394246+0000","last_unstale":"2024-06-03T01:08:04.272884+0000","last_undegraded":"2024-06-03T01:08:04.272884+0000","last_fullsized":"2024-06-03T01:08:04.272884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:48:47.331598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.264227+0000","last_change":"2024-06-03T01:04:54.378765+0000","last_active":"2024-06-03T01:08:04.264227+0000","last_peered":"2024-06-03T01:08:04.264227+0000","last_clean":"2024-06-03T01:08:04.264227+0000","last_became_active":"2024-06-03T01:04:54.378306+0000","last_became_peered":"2024-06-03T01:04:54.378306+0000","last_unstale":"2024-06-03T01:08:04.264227+0000","last_undegraded":"2024-06-03T01:08:04.264227+0000","last_fullsized":"2024-06-03T01:08:04.264227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:26:20.321196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.278742+0000","last_change":"2024-06-03T01:05:35.252114+0000","last_active":"2024-06-03T01:08:04.278742+0000","last_peered":"2024-06-03T01:08:04.278742+0000","last_clean":"2024-06-03T01:08:04.278742+0000","last_became_active":"2024-06-03T01:05:35.251849+0000","last_became_peered":"2024-06-03T01:05:35.251849+0000","last_unstale":"2024-06-03T01:08:04.278742+0000","last_undegraded":"2024-06-03T01:08:04.278742+0000","last_fullsized":"2024-06-03T01:08:04.278742+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:13:18.942723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.274537+0000","last_change":"2024-06-03T01:04:54.397423+0000","last_active":"2024-06-03T01:08:04.274537+0000","last_peered":"2024-06-03T01:08:04.274537+0000","last_clean":"2024-06-03T01:08:04.274537+0000","last_became_active":"2024-06-03T01:04:54.394329+0000","last_became_peered":"2024-06-03T01:04:54.394329+0000","last_unstale":"2024-06-03T01:08:04.274537+0000","last_undegraded":"2024-06-03T01:08:04.274537+0000","last_fullsized":"2024-06-03T01:08:04.274537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:04.020364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.260269+0000","last_change":"2024-06-03T01:04:54.368589+0000","last_active":"2024-06-03T01:08:04.260269+0000","last_peered":"2024-06-03T01:08:04.260269+0000","last_clean":"2024-06-03T01:08:04.260269+0000","last_became_active":"2024-06-03T01:04:54.368068+0000","last_became_peered":"2024-06-03T01:04:54.368068+0000","last_unstale":"2024-06-03T01:08:04.260269+0000","last_undegraded":"2024-06-03T01:08:04.260269+0000","last_fullsized":"2024-06-03T01:08:04.260269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:59:44.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.258394+0000","last_change":"2024-06-03T01:04:54.367488+0000","last_active":"2024-06-03T01:08:04.258394+0000","last_peered":"2024-06-03T01:08:04.258394+0000","last_clean":"2024-06-03T01:08:04.258394+0000","last_became_active":"2024-06-03T01:04:54.367022+0000","last_became_peered":"2024-06-03T01:04:54.367022+0000","last_unstale":"2024-06-03T01:08:04.258394+0000","last_undegraded":"2024-06-03T01:08:04.258394+0000","last_fullsized":"2024-06-03T01:08:04.258394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:52:27.697948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.268971+0000","last_change":"2024-06-03T01:04:54.366790+0000","last_active":"2024-06-03T01:08:04.268971+0000","last_peered":"2024-06-03T01:08:04.268971+0000","last_clean":"2024-06-03T01:08:04.268971+0000","last_became_active":"2024-06-03T01:04:54.366456+0000","last_became_peered":"2024-06-03T01:04:54.366456+0000","last_unstale":"2024-06-03T01:08:04.268971+0000","last_undegraded":"2024-06-03T01:08:04.268971+0000","last_fullsized":"2024-06-03T01:08:04.268971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:15:42.495284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.264768+0000","last_change":"2024-06-03T01:04:54.369483+0000","last_active":"2024-06-03T01:08:04.264768+0000","last_peered":"2024-06-03T01:08:04.264768+0000","last_clean":"2024-06-03T01:08:04.264768+0000","last_became_active":"2024-06-03T01:04:54.369118+0000","last_became_peered":"2024-06-03T01:04:54.369118+0000","last_unstale":"2024-06-03T01:08:04.264768+0000","last_undegraded":"2024-06-03T01:08:04.264768+0000","last_fullsized":"2024-06-03T01:08:04.264768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:18.218426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.285401+0000","last_change":"2024-06-03T01:04:54.380648+0000","last_active":"2024-06-03T01:08:04.285401+0000","last_peered":"2024-06-03T01:08:04.285401+0000","last_clean":"2024-06-03T01:08:04.285401+0000","last_became_active":"2024-06-03T01:04:54.380195+0000","last_became_peered":"2024-06-03T01:04:54.380195+0000","last_unstale":"2024-06-03T01:08:04.285401+0000","last_undegraded":"2024-06-03T01:08:04.285401+0000","last_fullsized":"2024-06-03T01:08:04.285401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:04.991858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290318+0000","last_change":"2024-06-03T01:04:54.367250+0000","last_active":"2024-06-03T01:08:04.290318+0000","last_peered":"2024-06-03T01:08:04.290318+0000","last_clean":"2024-06-03T01:08:04.290318+0000","last_became_active":"2024-06-03T01:04:54.364817+0000","last_became_peered":"2024-06-03T01:04:54.364817+0000","last_unstale":"2024-06-03T01:08:04.290318+0000","last_undegraded":"2024-06-03T01:08:04.290318+0000","last_fullsized":"2024-06-03T01:08:04.290318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:55:20.948692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.291050+0000","last_change":"2024-06-03T01:04:54.367640+0000","last_active":"2024-06-03T01:08:04.291050+0000","last_peered":"2024-06-03T01:08:04.291050+0000","last_clean":"2024-06-03T01:08:04.291050+0000","last_became_active":"2024-06-03T01:04:54.367080+0000","last_became_peered":"2024-06-03T01:04:54.367080+0000","last_unstale":"2024-06-03T01:08:04.291050+0000","last_undegraded":"2024-06-03T01:08:04.291050+0000","last_fullsized":"2024-06-03T01:08:04.291050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:43:23.469092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.293903+0000","last_change":"2024-06-03T01:04:54.376236+0000","last_active":"2024-06-03T01:08:04.293903+0000","last_peered":"2024-06-03T01:08:04.293903+0000","last_clean":"2024-06-03T01:08:04.293903+0000","last_became_active":"2024-06-03T01:04:54.375866+0000","last_became_peered":"2024-06-03T01:04:54.375866+0000","last_unstale":"2024-06-03T01:08:04.293903+0000","last_undegraded":"2024-06-03T01:08:04.293903+0000","last_fullsized":"2024-06-03T01:08:04.293903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:45.926391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.291713+0000","last_change":"2024-06-03T01:04:54.388824+0000","last_active":"2024-06-03T01:08:04.291713+0000","last_peered":"2024-06-03T01:08:04.291713+0000","last_clean":"2024-06-03T01:08:04.291713+0000","last_became_active":"2024-06-03T01:04:54.388491+0000","last_became_peered":"2024-06-03T01:04:54.388491+0000","last_unstale":"2024-06-03T01:08:04.291713+0000","last_undegraded":"2024-06-03T01:08:04.291713+0000","last_fullsized":"2024-06-03T01:08:04.291713+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:48.593126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.270633+0000","last_change":"2024-06-03T01:04:54.377623+0000","last_active":"2024-06-03T01:08:04.270633+0000","last_peered":"2024-06-03T01:08:04.270633+0000","last_clean":"2024-06-03T01:08:04.270633+0000","last_became_active":"2024-06-03T01:04:54.377037+0000","last_became_peered":"2024-06-03T01:04:54.377037+0000","last_unstale":"2024-06-03T01:08:04.270633+0000","last_undegraded":"2024-06-03T01:08:04.270633+0000","last_fullsized":"2024-06-03T01:08:04.270633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:52:12.729604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.261154+0000","last_change":"2024-06-03T01:04:54.396946+0000","last_active":"2024-06-03T01:08:04.261154+0000","last_peered":"2024-06-03T01:08:04.261154+0000","last_clean":"2024-06-03T01:08:04.261154+0000","last_became_active":"2024-06-03T01:04:54.396695+0000","last_became_peered":"2024-06-03T01:04:54.396695+0000","last_unstale":"2024-06-03T01:08:04.261154+0000","last_undegraded":"2024-06-03T01:08:04.261154+0000","last_fullsized":"2024-06-03T01:08:04.261154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:10:57.842403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347515+0000","last_change":"2024-06-03T01:05:08.399118+0000","last_active":"2024-06-03T01:06:35.347515+0000","last_peered":"2024-06-03T01:06:35.347515+0000","last_clean":"2024-06-03T01:06:35.347515+0000","last_became_active":"2024-06-03T01:05:08.398716+0000","last_became_peered":"2024-06-03T01:05:08.398716+0000","last_unstale":"2024-06-03T01:06:35.347515+0000","last_undegraded":"2024-06-03T01:06:35.347515+0000","last_fullsized":"2024-06-03T01:06:35.347515+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:28:46.911683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'14","reported_seq":186,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:23.621117+0000","last_change":"2024-06-03T01:04:49.781140+0000","last_active":"2024-06-03T01:08:23.621117+0000","last_peered":"2024-06-03T01:08:23.621117+0000","last_clean":"2024-06-03T01:08:23.621117+0000","last_became_active":"2024-06-03T01:04:49.780712+0000","last_became_peered":"2024-06-03T01:04:49.780712+0000","last_unstale":"2024-06-03T01:08:23.621117+0000","last_undegraded":"2024-06-03T01:08:23.621117+0000","last_fullsized":"2024-06-03T01:08:23.621117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:12:48.170997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.266137+0000","last_change":"2024-06-03T01:04:54.396470+0000","last_active":"2024-06-03T01:08:04.266137+0000","last_peered":"2024-06-03T01:08:04.266137+0000","last_clean":"2024-06-03T01:08:04.266137+0000","last_became_active":"2024-06-03T01:04:54.394564+0000","last_became_peered":"2024-06-03T01:04:54.394564+0000","last_unstale":"2024-06-03T01:08:04.266137+0000","last_undegraded":"2024-06-03T01:08:04.266137+0000","last_fullsized":"2024-06-03T01:08:04.266137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:21:06.436992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354702+0000","last_change":"2024-06-03T01:05:08.410302+0000","last_active":"2024-06-03T01:06:35.354702+0000","last_peered":"2024-06-03T01:06:35.354702+0000","last_clean":"2024-06-03T01:06:35.354702+0000","last_became_active":"2024-06-03T01:05:08.409971+0000","last_became_peered":"2024-06-03T01:05:08.409971+0000","last_unstale":"2024-06-03T01:06:35.354702+0000","last_undegraded":"2024-06-03T01:06:35.354702+0000","last_fullsized":"2024-06-03T01:06:35.354702+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:50.786491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.841198+0000","last_change":"2024-06-03T01:04:49.893771+0000","last_active":"2024-06-03T01:06:35.841198+0000","last_peered":"2024-06-03T01:06:35.841198+0000","last_clean":"2024-06-03T01:06:35.841198+0000","last_became_active":"2024-06-03T01:04:49.892855+0000","last_became_peered":"2024-06-03T01:04:49.892855+0000","last_unstale":"2024-06-03T01:06:35.841198+0000","last_undegraded":"2024-06-03T01:06:35.841198+0000","last_fullsized":"2024-06-03T01:06:35.841198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:47.474318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.296471+0000","last_change":"2024-06-03T01:04:54.378309+0000","last_active":"2024-06-03T01:08:04.296471+0000","last_peered":"2024-06-03T01:08:04.296471+0000","last_clean":"2024-06-03T01:08:04.296471+0000","last_became_active":"2024-06-03T01:04:54.377926+0000","last_became_peered":"2024-06-03T01:04:54.377926+0000","last_unstale":"2024-06-03T01:08:04.296471+0000","last_undegraded":"2024-06-03T01:08:04.296471+0000","last_fullsized":"2024-06-03T01:08:04.296471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:14:48.919904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.842782+0000","last_change":"2024-06-03T01:06:35.842264+0000","last_active":"2024-06-03T01:06:35.842782+0000","last_peered":"2024-06-03T01:06:35.842782+0000","last_clean":"2024-06-03T01:06:35.842782+0000","last_became_active":"2024-06-03T01:06:35.841786+0000","last_became_peered":"2024-06-03T01:06:35.841786+0000","last_unstale":"2024-06-03T01:06:35.842782+0000","last_undegraded":"2024-06-03T01:06:35.842782+0000","last_fullsized":"2024-06-03T01:06:35.842782+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:15.788405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":57,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.838007+0000","last_change":"2024-06-03T01:04:49.770978+0000","last_active":"2024-06-03T01:06:35.838007+0000","last_peered":"2024-06-03T01:06:35.838007+0000","last_clean":"2024-06-03T01:06:35.838007+0000","last_became_active":"2024-06-03T01:04:49.770010+0000","last_became_peered":"2024-06-03T01:04:49.770010+0000","last_unstale":"2024-06-03T01:06:35.838007+0000","last_undegraded":"2024-06-03T01:06:35.838007+0000","last_fullsized":"2024-06-03T01:06:35.838007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:32:33.707218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.276969+0000","last_change":"2024-06-03T01:04:54.380616+0000","last_active":"2024-06-03T01:08:04.276969+0000","last_peered":"2024-06-03T01:08:04.276969+0000","last_clean":"2024-06-03T01:08:04.276969+0000","last_became_active":"2024-06-03T01:04:54.380257+0000","last_became_peered":"2024-06-03T01:04:54.380257+0000","last_unstale":"2024-06-03T01:08:04.276969+0000","last_undegraded":"2024-06-03T01:08:04.276969+0000","last_fullsized":"2024-06-03T01:08:04.276969+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:14:54.327146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354695+0000","last_change":"2024-06-03T01:05:08.388139+0000","last_active":"2024-06-03T01:06:35.354695+0000","last_peered":"2024-06-03T01:06:35.354695+0000","last_clean":"2024-06-03T01:06:35.354695+0000","last_became_active":"2024-06-03T01:05:08.387697+0000","last_became_peered":"2024-06-03T01:05:08.387697+0000","last_unstale":"2024-06-03T01:06:35.354695+0000","last_undegraded":"2024-06-03T01:06:35.354695+0000","last_fullsized":"2024-06-03T01:06:35.354695+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:36:44.923801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837437+0000","last_change":"2024-06-03T01:04:49.771928+0000","last_active":"2024-06-03T01:06:35.837437+0000","last_peered":"2024-06-03T01:06:35.837437+0000","last_clean":"2024-06-03T01:06:35.837437+0000","last_became_active":"2024-06-03T01:04:49.771517+0000","last_became_peered":"2024-06-03T01:04:49.771517+0000","last_unstale":"2024-06-03T01:06:35.837437+0000","last_undegraded":"2024-06-03T01:06:35.837437+0000","last_fullsized":"2024-06-03T01:06:35.837437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:48:48.147276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":31,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.281239+0000","last_change":"2024-06-03T01:05:35.261543+0000","last_active":"2024-06-03T01:08:04.281239+0000","last_peered":"2024-06-03T01:08:04.281239+0000","last_clean":"2024-06-03T01:08:04.281239+0000","last_became_active":"2024-06-03T01:05:35.260977+0000","last_became_peered":"2024-06-03T01:05:35.260977+0000","last_unstale":"2024-06-03T01:08:04.281239+0000","last_undegraded":"2024-06-03T01:08:04.281239+0000","last_fullsized":"2024-06-03T01:08:04.281239+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:09:41.748477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.348031+0000","last_change":"2024-06-03T01:05:08.387864+0000","last_active":"2024-06-03T01:06:35.348031+0000","last_peered":"2024-06-03T01:06:35.348031+0000","last_clean":"2024-06-03T01:06:35.348031+0000","last_became_active":"2024-06-03T01:05:08.387381+0000","last_became_peered":"2024-06-03T01:05:08.387381+0000","last_unstale":"2024-06-03T01:06:35.348031+0000","last_undegraded":"2024-06-03T01:06:35.348031+0000","last_fullsized":"2024-06-03T01:06:35.348031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:18:31.349917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"100'7","reported_seq":182,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:23.516481+0000","last_change":"2024-06-03T01:04:49.770745+0000","last_active":"2024-06-03T01:08:23.516481+0000","last_peered":"2024-06-03T01:08:23.516481+0000","last_clean":"2024-06-03T01:08:23.516481+0000","last_became_active":"2024-06-03T01:04:49.770011+0000","last_became_peered":"2024-06-03T01:04:49.770011+0000","last_unstale":"2024-06-03T01:08:23.516481+0000","last_undegraded":"2024-06-03T01:08:23.516481+0000","last_fullsized":"2024-06-03T01:08:23.516481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:58:30.702656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.257080+0000","last_change":"2024-06-03T01:04:54.378551+0000","last_active":"2024-06-03T01:08:04.257080+0000","last_peered":"2024-06-03T01:08:04.257080+0000","last_clean":"2024-06-03T01:08:04.257080+0000","last_became_active":"2024-06-03T01:04:54.378201+0000","last_became_peered":"2024-06-03T01:04:54.378201+0000","last_unstale":"2024-06-03T01:08:04.257080+0000","last_undegraded":"2024-06-03T01:08:04.257080+0000","last_fullsized":"2024-06-03T01:08:04.257080+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:39:25.918280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347887+0000","last_change":"2024-06-03T01:05:08.407416+0000","last_active":"2024-06-03T01:06:35.347887+0000","last_peered":"2024-06-03T01:06:35.347887+0000","last_clean":"2024-06-03T01:06:35.347887+0000","last_became_active":"2024-06-03T01:05:08.406968+0000","last_became_peered":"2024-06-03T01:05:08.406968+0000","last_unstale":"2024-06-03T01:06:35.347887+0000","last_undegraded":"2024-06-03T01:06:35.347887+0000","last_fullsized":"2024-06-03T01:06:35.347887+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:35.618908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.363873+0000","last_change":"2024-06-03T01:04:49.772292+0000","last_active":"2024-06-03T01:06:35.363873+0000","last_peered":"2024-06-03T01:06:35.363873+0000","last_clean":"2024-06-03T01:06:35.363873+0000","last_became_active":"2024-06-03T01:04:49.771964+0000","last_became_peered":"2024-06-03T01:04:49.771964+0000","last_unstale":"2024-06-03T01:06:35.363873+0000","last_undegraded":"2024-06-03T01:06:35.363873+0000","last_fullsized":"2024-06-03T01:06:35.363873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:43:45.722433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.272773+0000","last_change":"2024-06-03T01:04:54.393864+0000","last_active":"2024-06-03T01:08:04.272773+0000","last_peered":"2024-06-03T01:08:04.272773+0000","last_clean":"2024-06-03T01:08:04.272773+0000","last_became_active":"2024-06-03T01:04:54.393442+0000","last_became_peered":"2024-06-03T01:04:54.393442+0000","last_unstale":"2024-06-03T01:08:04.272773+0000","last_undegraded":"2024-06-03T01:08:04.272773+0000","last_fullsized":"2024-06-03T01:08:04.272773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:29.910042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.599718+0000","last_change":"2024-06-03T01:06:35.599230+0000","last_active":"2024-06-03T01:06:35.599718+0000","last_peered":"2024-06-03T01:06:35.599718+0000","last_clean":"2024-06-03T01:06:35.599718+0000","last_became_active":"2024-06-03T01:06:35.598811+0000","last_became_peered":"2024-06-03T01:06:35.598811+0000","last_unstale":"2024-06-03T01:06:35.599718+0000","last_undegraded":"2024-06-03T01:06:35.599718+0000","last_fullsized":"2024-06-03T01:06:35.599718+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:14:50.943560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.350527+0000","last_change":"2024-06-03T01:04:49.781942+0000","last_active":"2024-06-03T01:06:35.350527+0000","last_peered":"2024-06-03T01:06:35.350527+0000","last_clean":"2024-06-03T01:06:35.350527+0000","last_became_active":"2024-06-03T01:04:49.781585+0000","last_became_peered":"2024-06-03T01:04:49.781585+0000","last_unstale":"2024-06-03T01:06:35.350527+0000","last_undegraded":"2024-06-03T01:06:35.350527+0000","last_fullsized":"2024-06-03T01:06:35.350527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:06:07.814196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.277836+0000","last_change":"2024-06-03T01:04:54.390073+0000","last_active":"2024-06-03T01:08:04.277836+0000","last_peered":"2024-06-03T01:08:04.277836+0000","last_clean":"2024-06-03T01:08:04.277836+0000","last_became_active":"2024-06-03T01:04:54.389698+0000","last_became_peered":"2024-06-03T01:04:54.389698+0000","last_unstale":"2024-06-03T01:08:04.277836+0000","last_undegraded":"2024-06-03T01:08:04.277836+0000","last_fullsized":"2024-06-03T01:08:04.277836+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:17:26.447432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597337+0000","last_change":"2024-06-03T01:05:08.395848+0000","last_active":"2024-06-03T01:06:35.597337+0000","last_peered":"2024-06-03T01:06:35.597337+0000","last_clean":"2024-06-03T01:06:35.597337+0000","last_became_active":"2024-06-03T01:05:08.394960+0000","last_became_peered":"2024-06-03T01:05:08.394960+0000","last_unstale":"2024-06-03T01:06:35.597337+0000","last_undegraded":"2024-06-03T01:06:35.597337+0000","last_fullsized":"2024-06-03T01:06:35.597337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:15.567354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366719+0000","last_change":"2024-06-03T01:04:49.770855+0000","last_active":"2024-06-03T01:06:35.366719+0000","last_peered":"2024-06-03T01:06:35.366719+0000","last_clean":"2024-06-03T01:06:35.366719+0000","last_became_active":"2024-06-03T01:04:49.770553+0000","last_became_peered":"2024-06-03T01:04:49.770553+0000","last_unstale":"2024-06-03T01:06:35.366719+0000","last_undegraded":"2024-06-03T01:06:35.366719+0000","last_fullsized":"2024-06-03T01:06:35.366719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:57.663765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.283973+0000","last_change":"2024-06-03T01:04:54.395962+0000","last_active":"2024-06-03T01:08:04.283973+0000","last_peered":"2024-06-03T01:08:04.283973+0000","last_clean":"2024-06-03T01:08:04.283973+0000","last_became_active":"2024-06-03T01:04:54.395463+0000","last_became_peered":"2024-06-03T01:04:54.395463+0000","last_unstale":"2024-06-03T01:08:04.283973+0000","last_undegraded":"2024-06-03T01:08:04.283973+0000","last_fullsized":"2024-06-03T01:08:04.283973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:21:48.570980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839998+0000","last_change":"2024-06-03T01:05:08.392196+0000","last_active":"2024-06-03T01:06:35.839998+0000","last_peered":"2024-06-03T01:06:35.839998+0000","last_clean":"2024-06-03T01:06:35.839998+0000","last_became_active":"2024-06-03T01:05:08.388980+0000","last_became_peered":"2024-06-03T01:05:08.388980+0000","last_unstale":"2024-06-03T01:06:35.839998+0000","last_undegraded":"2024-06-03T01:06:35.839998+0000","last_fullsized":"2024-06-03T01:06:35.839998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:01:01.066423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'6","reported_seq":299,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:21.513128+0000","last_change":"2024-06-03T01:04:49.785677+0000","last_active":"2024-06-03T01:08:21.513128+0000","last_peered":"2024-06-03T01:08:21.513128+0000","last_clean":"2024-06-03T01:08:21.513128+0000","last_became_active":"2024-06-03T01:04:49.785306+0000","last_became_peered":"2024-06-03T01:04:49.785306+0000","last_unstale":"2024-06-03T01:08:21.513128+0000","last_undegraded":"2024-06-03T01:08:21.513128+0000","last_fullsized":"2024-06-03T01:08:21.513128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:04:51.861728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":248,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.272212+0000","last_change":"2024-06-03T01:04:54.370886+0000","last_active":"2024-06-03T01:08:04.272212+0000","last_peered":"2024-06-03T01:08:04.272212+0000","last_clean":"2024-06-03T01:08:04.272212+0000","last_became_active":"2024-06-03T01:04:54.370555+0000","last_became_peered":"2024-06-03T01:04:54.370555+0000","last_unstale":"2024-06-03T01:08:04.272212+0000","last_undegraded":"2024-06-03T01:08:04.272212+0000","last_fullsized":"2024-06-03T01:08:04.272212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:20:23.802391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351608+0000","last_change":"2024-06-03T01:05:08.405735+0000","last_active":"2024-06-03T01:06:35.351608+0000","last_peered":"2024-06-03T01:06:35.351608+0000","last_clean":"2024-06-03T01:06:35.351608+0000","last_became_active":"2024-06-03T01:05:08.405289+0000","last_became_peered":"2024-06-03T01:05:08.405289+0000","last_unstale":"2024-06-03T01:06:35.351608+0000","last_undegraded":"2024-06-03T01:06:35.351608+0000","last_fullsized":"2024-06-03T01:06:35.351608+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:47:52.099470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382518+0000","last_change":"2024-06-03T01:04:49.771708+0000","last_active":"2024-06-03T01:06:35.382518+0000","last_peered":"2024-06-03T01:06:35.382518+0000","last_clean":"2024-06-03T01:06:35.382518+0000","last_became_active":"2024-06-03T01:04:49.771303+0000","last_became_peered":"2024-06-03T01:04:49.771303+0000","last_unstale":"2024-06-03T01:06:35.382518+0000","last_undegraded":"2024-06-03T01:06:35.382518+0000","last_fullsized":"2024-06-03T01:06:35.382518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:44:15.307453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.295154+0000","last_change":"2024-06-03T01:04:54.381206+0000","last_active":"2024-06-03T01:08:04.295154+0000","last_peered":"2024-06-03T01:08:04.295154+0000","last_clean":"2024-06-03T01:08:04.295154+0000","last_became_active":"2024-06-03T01:04:54.380842+0000","last_became_peered":"2024-06-03T01:04:54.380842+0000","last_unstale":"2024-06-03T01:08:04.295154+0000","last_undegraded":"2024-06-03T01:08:04.295154+0000","last_fullsized":"2024-06-03T01:08:04.295154+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:37.487354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355269+0000","last_change":"2024-06-03T01:05:08.402554+0000","last_active":"2024-06-03T01:06:35.355269+0000","last_peered":"2024-06-03T01:06:35.355269+0000","last_clean":"2024-06-03T01:06:35.355269+0000","last_became_active":"2024-06-03T01:05:08.402040+0000","last_became_peered":"2024-06-03T01:05:08.402040+0000","last_unstale":"2024-06-03T01:06:35.355269+0000","last_undegraded":"2024-06-03T01:06:35.355269+0000","last_fullsized":"2024-06-03T01:06:35.355269+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:10.713793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366984+0000","last_change":"2024-06-03T01:04:49.895941+0000","last_active":"2024-06-03T01:06:35.366984+0000","last_peered":"2024-06-03T01:06:35.366984+0000","last_clean":"2024-06-03T01:06:35.366984+0000","last_became_active":"2024-06-03T01:04:49.895579+0000","last_became_peered":"2024-06-03T01:04:49.895579+0000","last_unstale":"2024-06-03T01:06:35.366984+0000","last_undegraded":"2024-06-03T01:06:35.366984+0000","last_fullsized":"2024-06-03T01:06:35.366984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:45:31.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.264473+0000","last_change":"2024-06-03T01:04:54.369271+0000","last_active":"2024-06-03T01:08:04.264473+0000","last_peered":"2024-06-03T01:08:04.264473+0000","last_clean":"2024-06-03T01:08:04.264473+0000","last_became_active":"2024-06-03T01:04:54.368896+0000","last_became_peered":"2024-06-03T01:04:54.368896+0000","last_unstale":"2024-06-03T01:08:04.264473+0000","last_undegraded":"2024-06-03T01:08:04.264473+0000","last_fullsized":"2024-06-03T01:08:04.264473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:47:54.079642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351058+0000","last_change":"2024-06-03T01:05:08.406394+0000","last_active":"2024-06-03T01:06:35.351058+0000","last_peered":"2024-06-03T01:06:35.351058+0000","last_clean":"2024-06-03T01:06:35.351058+0000","last_became_active":"2024-06-03T01:05:08.405356+0000","last_became_peered":"2024-06-03T01:05:08.405356+0000","last_unstale":"2024-06-03T01:06:35.351058+0000","last_undegraded":"2024-06-03T01:06:35.351058+0000","last_fullsized":"2024-06-03T01:06:35.351058+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:57:40.648759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'42","reported_seq":345,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:21.512960+0000","last_change":"2024-06-03T01:04:49.771275+0000","last_active":"2024-06-03T01:08:21.512960+0000","last_peered":"2024-06-03T01:08:21.512960+0000","last_clean":"2024-06-03T01:08:21.512960+0000","last_became_active":"2024-06-03T01:04:49.770750+0000","last_became_peered":"2024-06-03T01:04:49.770750+0000","last_unstale":"2024-06-03T01:08:21.512960+0000","last_undegraded":"2024-06-03T01:08:21.512960+0000","last_fullsized":"2024-06-03T01:08:21.512960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:27:01.251134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68831,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":258,"num_read_kb":197,"num_write":44,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.257623+0000","last_change":"2024-06-03T01:04:54.364655+0000","last_active":"2024-06-03T01:08:04.257623+0000","last_peered":"2024-06-03T01:08:04.257623+0000","last_clean":"2024-06-03T01:08:04.257623+0000","last_became_active":"2024-06-03T01:04:54.364309+0000","last_became_peered":"2024-06-03T01:04:54.364309+0000","last_unstale":"2024-06-03T01:08:04.257623+0000","last_undegraded":"2024-06-03T01:08:04.257623+0000","last_fullsized":"2024-06-03T01:08:04.257623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:09:18.141328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.367298+0000","last_change":"2024-06-03T01:05:08.393014+0000","last_active":"2024-06-03T01:06:35.367298+0000","last_peered":"2024-06-03T01:06:35.367298+0000","last_clean":"2024-06-03T01:06:35.367298+0000","last_became_active":"2024-06-03T01:05:08.392614+0000","last_became_peered":"2024-06-03T01:05:08.392614+0000","last_unstale":"2024-06-03T01:06:35.367298+0000","last_undegraded":"2024-06-03T01:06:35.367298+0000","last_fullsized":"2024-06-03T01:06:35.367298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:23:36.625154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":56,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347784+0000","last_change":"2024-06-03T01:04:49.778551+0000","last_active":"2024-06-03T01:06:35.347784+0000","last_peered":"2024-06-03T01:06:35.347784+0000","last_clean":"2024-06-03T01:06:35.347784+0000","last_became_active":"2024-06-03T01:04:49.778166+0000","last_became_peered":"2024-06-03T01:04:49.778166+0000","last_unstale":"2024-06-03T01:06:35.347784+0000","last_undegraded":"2024-06-03T01:06:35.347784+0000","last_fullsized":"2024-06-03T01:06:35.347784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:02:40.207601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.273743+0000","last_change":"2024-06-03T01:04:54.394722+0000","last_active":"2024-06-03T01:08:04.273743+0000","last_peered":"2024-06-03T01:08:04.273743+0000","last_clean":"2024-06-03T01:08:04.273743+0000","last_became_active":"2024-06-03T01:04:54.393548+0000","last_became_peered":"2024-06-03T01:04:54.393548+0000","last_unstale":"2024-06-03T01:08:04.273743+0000","last_undegraded":"2024-06-03T01:08:04.273743+0000","last_fullsized":"2024-06-03T01:08:04.273743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:25:49.077880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354625+0000","last_change":"2024-06-03T01:05:08.403334+0000","last_active":"2024-06-03T01:06:35.354625+0000","last_peered":"2024-06-03T01:06:35.354625+0000","last_clean":"2024-06-03T01:06:35.354625+0000","last_became_active":"2024-06-03T01:05:08.402835+0000","last_became_peered":"2024-06-03T01:05:08.402835+0000","last_unstale":"2024-06-03T01:06:35.354625+0000","last_undegraded":"2024-06-03T01:06:35.354625+0000","last_fullsized":"2024-06-03T01:06:35.354625+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:50:08.551919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366736+0000","last_change":"2024-06-03T01:04:49.782860+0000","last_active":"2024-06-03T01:06:35.366736+0000","last_peered":"2024-06-03T01:06:35.366736+0000","last_clean":"2024-06-03T01:06:35.366736+0000","last_became_active":"2024-06-03T01:04:49.782610+0000","last_became_peered":"2024-06-03T01:04:49.782610+0000","last_unstale":"2024-06-03T01:06:35.366736+0000","last_undegraded":"2024-06-03T01:06:35.366736+0000","last_fullsized":"2024-06-03T01:06:35.366736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:37.967304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":59,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.285685+0000","last_change":"2024-06-03T01:05:35.253841+0000","last_active":"2024-06-03T01:08:04.285685+0000","last_peered":"2024-06-03T01:08:04.285685+0000","last_clean":"2024-06-03T01:08:04.285685+0000","last_became_active":"2024-06-03T01:05:35.253412+0000","last_became_peered":"2024-06-03T01:05:35.253412+0000","last_unstale":"2024-06-03T01:08:04.285685+0000","last_undegraded":"2024-06-03T01:08:04.285685+0000","last_fullsized":"2024-06-03T01:08:04.285685+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:12:21.890358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837806+0000","last_change":"2024-06-03T01:05:35.259007+0000","last_active":"2024-06-03T01:06:35.837806+0000","last_peered":"2024-06-03T01:06:35.837806+0000","last_clean":"2024-06-03T01:06:35.837806+0000","last_became_active":"2024-06-03T01:05:35.258358+0000","last_became_peered":"2024-06-03T01:05:35.258358+0000","last_unstale":"2024-06-03T01:06:35.837806+0000","last_undegraded":"2024-06-03T01:06:35.837806+0000","last_fullsized":"2024-06-03T01:06:35.837806+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:25:02.732772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"98'5","reported_seq":178,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:22.561288+0000","last_change":"2024-06-03T01:04:49.771410+0000","last_active":"2024-06-03T01:08:22.561288+0000","last_peered":"2024-06-03T01:08:22.561288+0000","last_clean":"2024-06-03T01:08:22.561288+0000","last_became_active":"2024-06-03T01:04:49.771034+0000","last_became_peered":"2024-06-03T01:04:49.771034+0000","last_unstale":"2024-06-03T01:08:22.561288+0000","last_undegraded":"2024-06-03T01:08:22.561288+0000","last_fullsized":"2024-06-03T01:08:22.561288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:25:30.888075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.266180+0000","last_change":"2024-06-03T01:04:54.363918+0000","last_active":"2024-06-03T01:08:04.266180+0000","last_peered":"2024-06-03T01:08:04.266180+0000","last_clean":"2024-06-03T01:08:04.266180+0000","last_became_active":"2024-06-03T01:04:54.363536+0000","last_became_peered":"2024-06-03T01:04:54.363536+0000","last_unstale":"2024-06-03T01:08:04.266180+0000","last_undegraded":"2024-06-03T01:08:04.266180+0000","last_fullsized":"2024-06-03T01:08:04.266180+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:28:22.103375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.353924+0000","last_change":"2024-06-03T01:05:08.403245+0000","last_active":"2024-06-03T01:06:35.353924+0000","last_peered":"2024-06-03T01:06:35.353924+0000","last_clean":"2024-06-03T01:06:35.353924+0000","last_became_active":"2024-06-03T01:05:08.402755+0000","last_became_peered":"2024-06-03T01:05:08.402755+0000","last_unstale":"2024-06-03T01:06:35.353924+0000","last_undegraded":"2024-06-03T01:06:35.353924+0000","last_fullsized":"2024-06-03T01:06:35.353924+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:33.777084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837123+0000","last_change":"2024-06-03T01:04:49.891326+0000","last_active":"2024-06-03T01:06:35.837123+0000","last_peered":"2024-06-03T01:06:35.837123+0000","last_clean":"2024-06-03T01:06:35.837123+0000","last_became_active":"2024-06-03T01:04:49.891040+0000","last_became_peered":"2024-06-03T01:04:49.891040+0000","last_unstale":"2024-06-03T01:06:35.837123+0000","last_undegraded":"2024-06-03T01:06:35.837123+0000","last_fullsized":"2024-06-03T01:06:35.837123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:39:26.896996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.258488+0000","last_change":"2024-06-03T01:04:54.372779+0000","last_active":"2024-06-03T01:08:04.258488+0000","last_peered":"2024-06-03T01:08:04.258488+0000","last_clean":"2024-06-03T01:08:04.258488+0000","last_became_active":"2024-06-03T01:04:54.372427+0000","last_became_peered":"2024-06-03T01:04:54.372427+0000","last_unstale":"2024-06-03T01:08:04.258488+0000","last_undegraded":"2024-06-03T01:08:04.258488+0000","last_fullsized":"2024-06-03T01:08:04.258488+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:21:18.837300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366565+0000","last_change":"2024-06-03T01:05:08.391281+0000","last_active":"2024-06-03T01:06:35.366565+0000","last_peered":"2024-06-03T01:06:35.366565+0000","last_clean":"2024-06-03T01:06:35.366565+0000","last_became_active":"2024-06-03T01:05:08.390871+0000","last_became_peered":"2024-06-03T01:05:08.390871+0000","last_unstale":"2024-06-03T01:06:35.366565+0000","last_undegraded":"2024-06-03T01:06:35.366565+0000","last_fullsized":"2024-06-03T01:06:35.366565+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:19:12.174065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837965+0000","last_change":"2024-06-03T01:04:49.769587+0000","last_active":"2024-06-03T01:06:35.837965+0000","last_peered":"2024-06-03T01:06:35.837965+0000","last_clean":"2024-06-03T01:06:35.837965+0000","last_became_active":"2024-06-03T01:04:49.769267+0000","last_became_peered":"2024-06-03T01:04:49.769267+0000","last_unstale":"2024-06-03T01:06:35.837965+0000","last_undegraded":"2024-06-03T01:06:35.837965+0000","last_fullsized":"2024-06-03T01:06:35.837965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:18:30.747021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.254728+0000","last_change":"2024-06-03T01:04:54.368293+0000","last_active":"2024-06-03T01:08:04.254728+0000","last_peered":"2024-06-03T01:08:04.254728+0000","last_clean":"2024-06-03T01:08:04.254728+0000","last_became_active":"2024-06-03T01:04:54.367910+0000","last_became_peered":"2024-06-03T01:04:54.367910+0000","last_unstale":"2024-06-03T01:08:04.254728+0000","last_undegraded":"2024-06-03T01:08:04.254728+0000","last_fullsized":"2024-06-03T01:08:04.254728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:23:46.977133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837179+0000","last_change":"2024-06-03T01:05:08.619467+0000","last_active":"2024-06-03T01:06:35.837179+0000","last_peered":"2024-06-03T01:06:35.837179+0000","last_clean":"2024-06-03T01:06:35.837179+0000","last_became_active":"2024-06-03T01:05:08.618884+0000","last_became_peered":"2024-06-03T01:05:08.618884+0000","last_unstale":"2024-06-03T01:06:35.837179+0000","last_undegraded":"2024-06-03T01:06:35.837179+0000","last_fullsized":"2024-06-03T01:06:35.837179+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:22:22.072920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352552+0000","last_change":"2024-06-03T01:04:49.896429+0000","last_active":"2024-06-03T01:06:35.352552+0000","last_peered":"2024-06-03T01:06:35.352552+0000","last_clean":"2024-06-03T01:06:35.352552+0000","last_became_active":"2024-06-03T01:04:49.895957+0000","last_became_peered":"2024-06-03T01:04:49.895957+0000","last_unstale":"2024-06-03T01:06:35.352552+0000","last_undegraded":"2024-06-03T01:06:35.352552+0000","last_fullsized":"2024-06-03T01:06:35.352552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:17:52.362276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.269606+0000","last_change":"2024-06-03T01:04:54.396972+0000","last_active":"2024-06-03T01:08:04.269606+0000","last_peered":"2024-06-03T01:08:04.269606+0000","last_clean":"2024-06-03T01:08:04.269606+0000","last_became_active":"2024-06-03T01:04:54.396737+0000","last_became_peered":"2024-06-03T01:04:54.396737+0000","last_unstale":"2024-06-03T01:08:04.269606+0000","last_undegraded":"2024-06-03T01:08:04.269606+0000","last_fullsized":"2024-06-03T01:08:04.269606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:38:31.722765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.351660+0000","last_change":"2024-06-03T01:05:08.390129+0000","last_active":"2024-06-03T01:06:35.351660+0000","last_peered":"2024-06-03T01:06:35.351660+0000","last_clean":"2024-06-03T01:06:35.351660+0000","last_became_active":"2024-06-03T01:05:08.389553+0000","last_became_peered":"2024-06-03T01:05:08.389553+0000","last_unstale":"2024-06-03T01:06:35.351660+0000","last_undegraded":"2024-06-03T01:06:35.351660+0000","last_fullsized":"2024-06-03T01:06:35.351660+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:53:34.176420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"100'7","reported_seq":180,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:22.619901+0000","last_change":"2024-06-03T01:04:49.777579+0000","last_active":"2024-06-03T01:08:22.619901+0000","last_peered":"2024-06-03T01:08:22.619901+0000","last_clean":"2024-06-03T01:08:22.619901+0000","last_became_active":"2024-06-03T01:04:49.777207+0000","last_became_peered":"2024-06-03T01:04:49.777207+0000","last_unstale":"2024-06-03T01:08:22.619901+0000","last_undegraded":"2024-06-03T01:08:22.619901+0000","last_fullsized":"2024-06-03T01:08:22.619901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:59:57.941162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.289556+0000","last_change":"2024-06-03T01:04:54.382883+0000","last_active":"2024-06-03T01:08:04.289556+0000","last_peered":"2024-06-03T01:08:04.289556+0000","last_clean":"2024-06-03T01:08:04.289556+0000","last_became_active":"2024-06-03T01:04:54.382445+0000","last_became_peered":"2024-06-03T01:04:54.382445+0000","last_unstale":"2024-06-03T01:08:04.289556+0000","last_undegraded":"2024-06-03T01:08:04.289556+0000","last_fullsized":"2024-06-03T01:08:04.289556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:47:07.549646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354476+0000","last_change":"2024-06-03T01:05:08.619324+0000","last_active":"2024-06-03T01:06:35.354476+0000","last_peered":"2024-06-03T01:06:35.354476+0000","last_clean":"2024-06-03T01:06:35.354476+0000","last_became_active":"2024-06-03T01:05:08.618856+0000","last_became_peered":"2024-06-03T01:05:08.618856+0000","last_unstale":"2024-06-03T01:06:35.354476+0000","last_undegraded":"2024-06-03T01:06:35.354476+0000","last_fullsized":"2024-06-03T01:06:35.354476+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:55:49.520528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.260177+0000","last_change":"2024-06-03T01:04:54.371883+0000","last_active":"2024-06-03T01:08:04.260177+0000","last_peered":"2024-06-03T01:08:04.260177+0000","last_clean":"2024-06-03T01:08:04.260177+0000","last_became_active":"2024-06-03T01:04:54.371502+0000","last_became_peered":"2024-06-03T01:04:54.371502+0000","last_unstale":"2024-06-03T01:08:04.260177+0000","last_undegraded":"2024-06-03T01:08:04.260177+0000","last_fullsized":"2024-06-03T01:08:04.260177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:40:14.632927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":176,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:22.560602+0000","last_change":"2024-06-03T01:04:49.893131+0000","last_active":"2024-06-03T01:08:22.560602+0000","last_peered":"2024-06-03T01:08:22.560602+0000","last_clean":"2024-06-03T01:08:22.560602+0000","last_became_active":"2024-06-03T01:04:49.892122+0000","last_became_peered":"2024-06-03T01:04:49.892122+0000","last_unstale":"2024-06-03T01:08:22.560602+0000","last_undegraded":"2024-06-03T01:08:22.560602+0000","last_fullsized":"2024-06-03T01:08:22.560602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:09:07.852255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382609+0000","last_change":"2024-06-03T01:05:08.399018+0000","last_active":"2024-06-03T01:06:35.382609+0000","last_peered":"2024-06-03T01:06:35.382609+0000","last_clean":"2024-06-03T01:06:35.382609+0000","last_became_active":"2024-06-03T01:05:08.398531+0000","last_became_peered":"2024-06-03T01:05:08.398531+0000","last_unstale":"2024-06-03T01:06:35.382609+0000","last_undegraded":"2024-06-03T01:06:35.382609+0000","last_fullsized":"2024-06-03T01:06:35.382609+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:25:54.115344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.267539+0000","last_change":"2024-06-03T01:04:54.385215+0000","last_active":"2024-06-03T01:08:04.267539+0000","last_peered":"2024-06-03T01:08:04.267539+0000","last_clean":"2024-06-03T01:08:04.267539+0000","last_became_active":"2024-06-03T01:04:54.384764+0000","last_became_peered":"2024-06-03T01:04:54.384764+0000","last_unstale":"2024-06-03T01:08:04.267539+0000","last_undegraded":"2024-06-03T01:08:04.267539+0000","last_fullsized":"2024-06-03T01:08:04.267539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:05:45.388604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354880+0000","last_change":"2024-06-03T01:04:49.781444+0000","last_active":"2024-06-03T01:06:35.354880+0000","last_peered":"2024-06-03T01:06:35.354880+0000","last_clean":"2024-06-03T01:06:35.354880+0000","last_became_active":"2024-06-03T01:04:49.780922+0000","last_became_peered":"2024-06-03T01:04:49.780922+0000","last_unstale":"2024-06-03T01:06:35.354880+0000","last_undegraded":"2024-06-03T01:06:35.354880+0000","last_fullsized":"2024-06-03T01:06:35.354880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:19.843289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":379,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382057+0000","last_change":"2024-06-03T01:03:16.011282+0000","last_active":"2024-06-03T01:06:35.382057+0000","last_peered":"2024-06-03T01:06:35.382057+0000","last_clean":"2024-06-03T01:06:35.382057+0000","last_became_active":"2024-06-03T01:03:16.010726+0000","last_became_peered":"2024-06-03T01:03:16.010726+0000","last_unstale":"2024-06-03T01:06:35.382057+0000","last_undegraded":"2024-06-03T01:06:35.382057+0000","last_fullsized":"2024-06-03T01:06:35.382057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T00:59:20.780015+0000","last_clean_scrub_stamp":"2024-06-03T00:59:20.780015+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:50:32.649940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.367062+0000","last_change":"2024-06-03T01:05:08.620410+0000","last_active":"2024-06-03T01:06:35.367062+0000","last_peered":"2024-06-03T01:06:35.367062+0000","last_clean":"2024-06-03T01:06:35.367062+0000","last_became_active":"2024-06-03T01:05:08.619601+0000","last_became_peered":"2024-06-03T01:05:08.619601+0000","last_unstale":"2024-06-03T01:06:35.367062+0000","last_undegraded":"2024-06-03T01:06:35.367062+0000","last_fullsized":"2024-06-03T01:06:35.367062+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:15:47.073077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.252417+0000","last_change":"2024-06-03T01:04:54.379373+0000","last_active":"2024-06-03T01:08:04.252417+0000","last_peered":"2024-06-03T01:08:04.252417+0000","last_clean":"2024-06-03T01:08:04.252417+0000","last_became_active":"2024-06-03T01:04:54.378981+0000","last_became_peered":"2024-06-03T01:04:54.378981+0000","last_unstale":"2024-06-03T01:08:04.252417+0000","last_undegraded":"2024-06-03T01:08:04.252417+0000","last_fullsized":"2024-06-03T01:08:04.252417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:27:06.651091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.382211+0000","last_change":"2024-06-03T01:04:49.773896+0000","last_active":"2024-06-03T01:06:35.382211+0000","last_peered":"2024-06-03T01:06:35.382211+0000","last_clean":"2024-06-03T01:06:35.382211+0000","last_became_active":"2024-06-03T01:04:49.773403+0000","last_became_peered":"2024-06-03T01:04:49.773403+0000","last_unstale":"2024-06-03T01:06:35.382211+0000","last_undegraded":"2024-06-03T01:06:35.382211+0000","last_fullsized":"2024-06-03T01:06:35.382211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:34:13.376589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354565+0000","last_change":"2024-06-03T01:05:08.389575+0000","last_active":"2024-06-03T01:06:35.354565+0000","last_peered":"2024-06-03T01:06:35.354565+0000","last_clean":"2024-06-03T01:06:35.354565+0000","last_became_active":"2024-06-03T01:05:08.389010+0000","last_became_peered":"2024-06-03T01:05:08.389010+0000","last_unstale":"2024-06-03T01:06:35.354565+0000","last_undegraded":"2024-06-03T01:06:35.354565+0000","last_fullsized":"2024-06-03T01:06:35.354565+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T12:57:26.861892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354714+0000","last_change":"2024-06-03T01:05:35.263346+0000","last_active":"2024-06-03T01:06:35.354714+0000","last_peered":"2024-06-03T01:06:35.354714+0000","last_clean":"2024-06-03T01:06:35.354714+0000","last_became_active":"2024-06-03T01:05:35.262843+0000","last_became_peered":"2024-06-03T01:05:35.262843+0000","last_unstale":"2024-06-03T01:06:35.354714+0000","last_undegraded":"2024-06-03T01:06:35.354714+0000","last_fullsized":"2024-06-03T01:06:35.354714+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:44.247086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.283246+0000","last_change":"2024-06-03T01:04:54.388176+0000","last_active":"2024-06-03T01:08:04.283246+0000","last_peered":"2024-06-03T01:08:04.283246+0000","last_clean":"2024-06-03T01:08:04.283246+0000","last_became_active":"2024-06-03T01:04:54.384544+0000","last_became_peered":"2024-06-03T01:04:54.384544+0000","last_unstale":"2024-06-03T01:08:04.283246+0000","last_undegraded":"2024-06-03T01:08:04.283246+0000","last_fullsized":"2024-06-03T01:08:04.283246+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:00:42.931031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.839686+0000","last_change":"2024-06-03T01:05:08.619540+0000","last_active":"2024-06-03T01:06:35.839686+0000","last_peered":"2024-06-03T01:06:35.839686+0000","last_clean":"2024-06-03T01:06:35.839686+0000","last_became_active":"2024-06-03T01:05:08.618834+0000","last_became_peered":"2024-06-03T01:05:08.618834+0000","last_unstale":"2024-06-03T01:06:35.839686+0000","last_undegraded":"2024-06-03T01:06:35.839686+0000","last_fullsized":"2024-06-03T01:06:35.839686+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:07:05.467161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'17","reported_seq":61,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:07:00.330523+0000","last_change":"2024-06-03T01:04:49.786635+0000","last_active":"2024-06-03T01:07:00.330523+0000","last_peered":"2024-06-03T01:07:00.330523+0000","last_clean":"2024-06-03T01:07:00.330523+0000","last_became_active":"2024-06-03T01:04:49.785951+0000","last_became_peered":"2024-06-03T01:04:49.785951+0000","last_unstale":"2024-06-03T01:07:00.330523+0000","last_undegraded":"2024-06-03T01:07:00.330523+0000","last_fullsized":"2024-06-03T01:07:00.330523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:10.784322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.273463+0000","last_change":"2024-06-03T01:04:54.387111+0000","last_active":"2024-06-03T01:08:04.273463+0000","last_peered":"2024-06-03T01:08:04.273463+0000","last_clean":"2024-06-03T01:08:04.273463+0000","last_became_active":"2024-06-03T01:04:54.384691+0000","last_became_peered":"2024-06-03T01:04:54.384691+0000","last_unstale":"2024-06-03T01:08:04.273463+0000","last_undegraded":"2024-06-03T01:08:04.273463+0000","last_fullsized":"2024-06-03T01:08:04.273463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:11:15.254855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347498+0000","last_change":"2024-06-03T01:05:08.386335+0000","last_active":"2024-06-03T01:06:35.347498+0000","last_peered":"2024-06-03T01:06:35.347498+0000","last_clean":"2024-06-03T01:06:35.347498+0000","last_became_active":"2024-06-03T01:05:08.385722+0000","last_became_peered":"2024-06-03T01:05:08.385722+0000","last_unstale":"2024-06-03T01:06:35.347498+0000","last_undegraded":"2024-06-03T01:06:35.347498+0000","last_fullsized":"2024-06-03T01:06:35.347498+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:09:00.918031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355343+0000","last_change":"2024-06-03T01:04:49.883125+0000","last_active":"2024-06-03T01:06:35.355343+0000","last_peered":"2024-06-03T01:06:35.355343+0000","last_clean":"2024-06-03T01:06:35.355343+0000","last_became_active":"2024-06-03T01:04:49.882760+0000","last_became_peered":"2024-06-03T01:04:49.882760+0000","last_unstale":"2024-06-03T01:06:35.355343+0000","last_undegraded":"2024-06-03T01:06:35.355343+0000","last_fullsized":"2024-06-03T01:06:35.355343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:05:30.854654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290667+0000","last_change":"2024-06-03T01:04:54.386265+0000","last_active":"2024-06-03T01:08:04.290667+0000","last_peered":"2024-06-03T01:08:04.290667+0000","last_clean":"2024-06-03T01:08:04.290667+0000","last_became_active":"2024-06-03T01:04:54.385801+0000","last_became_peered":"2024-06-03T01:04:54.385801+0000","last_unstale":"2024-06-03T01:08:04.290667+0000","last_undegraded":"2024-06-03T01:08:04.290667+0000","last_fullsized":"2024-06-03T01:08:04.290667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:40:26.128516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355206+0000","last_change":"2024-06-03T01:05:08.619166+0000","last_active":"2024-06-03T01:06:35.355206+0000","last_peered":"2024-06-03T01:06:35.355206+0000","last_clean":"2024-06-03T01:06:35.355206+0000","last_became_active":"2024-06-03T01:05:08.618601+0000","last_became_peered":"2024-06-03T01:05:08.618601+0000","last_unstale":"2024-06-03T01:06:35.355206+0000","last_undegraded":"2024-06-03T01:06:35.355206+0000","last_fullsized":"2024-06-03T01:06:35.355206+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:55:32.401683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837348+0000","last_change":"2024-06-03T01:05:35.258905+0000","last_active":"2024-06-03T01:06:35.837348+0000","last_peered":"2024-06-03T01:06:35.837348+0000","last_clean":"2024-06-03T01:06:35.837348+0000","last_became_active":"2024-06-03T01:05:35.258423+0000","last_became_peered":"2024-06-03T01:05:35.258423+0000","last_unstale":"2024-06-03T01:06:35.837348+0000","last_undegraded":"2024-06-03T01:06:35.837348+0000","last_fullsized":"2024-06-03T01:06:35.837348+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:31:36.969581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.222910+0000","last_change":"2024-06-03T01:04:54.367788+0000","last_active":"2024-06-03T01:08:04.222910+0000","last_peered":"2024-06-03T01:08:04.222910+0000","last_clean":"2024-06-03T01:08:04.222910+0000","last_became_active":"2024-06-03T01:04:54.367353+0000","last_became_peered":"2024-06-03T01:04:54.367353+0000","last_unstale":"2024-06-03T01:08:04.222910+0000","last_undegraded":"2024-06-03T01:08:04.222910+0000","last_fullsized":"2024-06-03T01:08:04.222910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:39:26.604176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.352869+0000","last_change":"2024-06-03T01:05:08.405875+0000","last_active":"2024-06-03T01:06:35.352869+0000","last_peered":"2024-06-03T01:06:35.352869+0000","last_clean":"2024-06-03T01:06:35.352869+0000","last_became_active":"2024-06-03T01:05:08.405410+0000","last_became_peered":"2024-06-03T01:05:08.405410+0000","last_unstale":"2024-06-03T01:06:35.352869+0000","last_undegraded":"2024-06-03T01:06:35.352869+0000","last_fullsized":"2024-06-03T01:06:35.352869+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:19:51.467150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.840215+0000","last_change":"2024-06-03T01:04:49.786446+0000","last_active":"2024-06-03T01:06:35.840215+0000","last_peered":"2024-06-03T01:06:35.840215+0000","last_clean":"2024-06-03T01:06:35.840215+0000","last_became_active":"2024-06-03T01:04:49.786036+0000","last_became_peered":"2024-06-03T01:04:49.786036+0000","last_unstale":"2024-06-03T01:06:35.840215+0000","last_undegraded":"2024-06-03T01:06:35.840215+0000","last_fullsized":"2024-06-03T01:06:35.840215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:41:12.176583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288101+0000","last_change":"2024-06-03T01:04:54.382742+0000","last_active":"2024-06-03T01:08:04.288101+0000","last_peered":"2024-06-03T01:08:04.288101+0000","last_clean":"2024-06-03T01:08:04.288101+0000","last_became_active":"2024-06-03T01:04:54.382296+0000","last_became_peered":"2024-06-03T01:04:54.382296+0000","last_unstale":"2024-06-03T01:08:04.288101+0000","last_undegraded":"2024-06-03T01:08:04.288101+0000","last_fullsized":"2024-06-03T01:08:04.288101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:32:03.385387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"100'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:39.228380+0000","last_change":"2024-06-03T01:05:08.403233+0000","last_active":"2024-06-03T01:06:39.228380+0000","last_peered":"2024-06-03T01:06:39.228380+0000","last_clean":"2024-06-03T01:06:39.228380+0000","last_became_active":"2024-06-03T01:05:08.402693+0000","last_became_peered":"2024-06-03T01:05:08.402693+0000","last_unstale":"2024-06-03T01:06:39.228380+0000","last_undegraded":"2024-06-03T01:06:39.228380+0000","last_fullsized":"2024-06-03T01:06:39.228380+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:35:30.722557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.347626+0000","last_change":"2024-06-03T01:04:49.779089+0000","last_active":"2024-06-03T01:06:35.347626+0000","last_peered":"2024-06-03T01:06:35.347626+0000","last_clean":"2024-06-03T01:06:35.347626+0000","last_became_active":"2024-06-03T01:04:49.778162+0000","last_became_peered":"2024-06-03T01:04:49.778162+0000","last_unstale":"2024-06-03T01:06:35.347626+0000","last_undegraded":"2024-06-03T01:06:35.347626+0000","last_fullsized":"2024-06-03T01:06:35.347626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:44:43.151731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.291790+0000","last_change":"2024-06-03T01:04:54.393447+0000","last_active":"2024-06-03T01:08:04.291790+0000","last_peered":"2024-06-03T01:08:04.291790+0000","last_clean":"2024-06-03T01:08:04.291790+0000","last_became_active":"2024-06-03T01:04:54.390154+0000","last_became_peered":"2024-06-03T01:04:54.390154+0000","last_unstale":"2024-06-03T01:08:04.291790+0000","last_undegraded":"2024-06-03T01:08:04.291790+0000","last_fullsized":"2024-06-03T01:08:04.291790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:00:21.854391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.355124+0000","last_change":"2024-06-03T01:05:08.619603+0000","last_active":"2024-06-03T01:06:35.355124+0000","last_peered":"2024-06-03T01:06:35.355124+0000","last_clean":"2024-06-03T01:06:35.355124+0000","last_became_active":"2024-06-03T01:05:08.618554+0000","last_became_peered":"2024-06-03T01:05:08.618554+0000","last_unstale":"2024-06-03T01:06:35.355124+0000","last_undegraded":"2024-06-03T01:06:35.355124+0000","last_fullsized":"2024-06-03T01:06:35.355124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:57:26.125390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837454+0000","last_change":"2024-06-03T01:04:49.770531+0000","last_active":"2024-06-03T01:06:35.837454+0000","last_peered":"2024-06-03T01:06:35.837454+0000","last_clean":"2024-06-03T01:06:35.837454+0000","last_became_active":"2024-06-03T01:04:49.770217+0000","last_became_peered":"2024-06-03T01:04:49.770217+0000","last_unstale":"2024-06-03T01:06:35.837454+0000","last_undegraded":"2024-06-03T01:06:35.837454+0000","last_fullsized":"2024-06-03T01:06:35.837454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:59:52.181262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.277547+0000","last_change":"2024-06-03T01:04:54.376511+0000","last_active":"2024-06-03T01:08:04.277547+0000","last_peered":"2024-06-03T01:08:04.277547+0000","last_clean":"2024-06-03T01:08:04.277547+0000","last_became_active":"2024-06-03T01:04:54.375896+0000","last_became_peered":"2024-06-03T01:04:54.375896+0000","last_unstale":"2024-06-03T01:08:04.277547+0000","last_undegraded":"2024-06-03T01:08:04.277547+0000","last_fullsized":"2024-06-03T01:08:04.277547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:20.237450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.837429+0000","last_change":"2024-06-03T01:05:08.387551+0000","last_active":"2024-06-03T01:06:35.837429+0000","last_peered":"2024-06-03T01:06:35.837429+0000","last_clean":"2024-06-03T01:06:35.837429+0000","last_became_active":"2024-06-03T01:05:08.386719+0000","last_became_peered":"2024-06-03T01:05:08.386719+0000","last_unstale":"2024-06-03T01:06:35.837429+0000","last_undegraded":"2024-06-03T01:06:35.837429+0000","last_fullsized":"2024-06-03T01:06:35.837429+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:03:57.131330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":183,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:23.608996+0000","last_change":"2024-06-03T01:04:49.890435+0000","last_active":"2024-06-03T01:08:23.608996+0000","last_peered":"2024-06-03T01:08:23.608996+0000","last_clean":"2024-06-03T01:08:23.608996+0000","last_became_active":"2024-06-03T01:04:49.890074+0000","last_became_peered":"2024-06-03T01:04:49.890074+0000","last_unstale":"2024-06-03T01:08:23.608996+0000","last_undegraded":"2024-06-03T01:08:23.608996+0000","last_fullsized":"2024-06-03T01:08:23.608996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:48:46.743934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.268967+0000","last_change":"2024-06-03T01:04:54.366487+0000","last_active":"2024-06-03T01:08:04.268967+0000","last_peered":"2024-06-03T01:08:04.268967+0000","last_clean":"2024-06-03T01:08:04.268967+0000","last_became_active":"2024-06-03T01:04:54.365905+0000","last_became_peered":"2024-06-03T01:04:54.365905+0000","last_unstale":"2024-06-03T01:08:04.268967+0000","last_undegraded":"2024-06-03T01:08:04.268967+0000","last_fullsized":"2024-06-03T01:08:04.268967+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T13:04:09.159028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.597029+0000","last_change":"2024-06-03T01:05:08.404608+0000","last_active":"2024-06-03T01:06:35.597029+0000","last_peered":"2024-06-03T01:06:35.597029+0000","last_clean":"2024-06-03T01:06:35.597029+0000","last_became_active":"2024-06-03T01:05:08.404148+0000","last_became_peered":"2024-06-03T01:05:08.404148+0000","last_unstale":"2024-06-03T01:06:35.597029+0000","last_undegraded":"2024-06-03T01:06:35.597029+0000","last_fullsized":"2024-06-03T01:06:35.597029+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:29:54.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366988+0000","last_change":"2024-06-03T01:04:49.778805+0000","last_active":"2024-06-03T01:06:35.366988+0000","last_peered":"2024-06-03T01:06:35.366988+0000","last_clean":"2024-06-03T01:06:35.366988+0000","last_became_active":"2024-06-03T01:04:49.778436+0000","last_became_peered":"2024-06-03T01:04:49.778436+0000","last_unstale":"2024-06-03T01:06:35.366988+0000","last_undegraded":"2024-06-03T01:06:35.366988+0000","last_fullsized":"2024-06-03T01:06:35.366988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:21.369521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.271058+0000","last_change":"2024-06-03T01:04:54.393604+0000","last_active":"2024-06-03T01:08:04.271058+0000","last_peered":"2024-06-03T01:08:04.271058+0000","last_clean":"2024-06-03T01:08:04.271058+0000","last_became_active":"2024-06-03T01:04:54.393282+0000","last_became_peered":"2024-06-03T01:04:54.393282+0000","last_unstale":"2024-06-03T01:08:04.271058+0000","last_undegraded":"2024-06-03T01:08:04.271058+0000","last_fullsized":"2024-06-03T01:08:04.271058+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:17:24.435502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.366426+0000","last_change":"2024-06-03T01:05:08.386399+0000","last_active":"2024-06-03T01:06:35.366426+0000","last_peered":"2024-06-03T01:06:35.366426+0000","last_clean":"2024-06-03T01:06:35.366426+0000","last_became_active":"2024-06-03T01:05:08.384547+0000","last_became_peered":"2024-06-03T01:05:08.384547+0000","last_unstale":"2024-06-03T01:06:35.366426+0000","last_undegraded":"2024-06-03T01:06:35.366426+0000","last_fullsized":"2024-06-03T01:06:35.366426+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:05:07.335194+0000","last_clean_scrub_stamp":"2024-06-03T01:05:07.335194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:04:17.097695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:06:35.354216+0000","last_change":"2024-06-03T01:04:49.782877+0000","last_active":"2024-06-03T01:06:35.354216+0000","last_peered":"2024-06-03T01:06:35.354216+0000","last_clean":"2024-06-03T01:06:35.354216+0000","last_became_active":"2024-06-03T01:04:49.781399+0000","last_became_peered":"2024-06-03T01:04:49.781399+0000","last_unstale":"2024-06-03T01:06:35.354216+0000","last_undegraded":"2024-06-03T01:06:35.354216+0000","last_fullsized":"2024-06-03T01:06:35.354216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:48.749134+0000","last_clean_scrub_stamp":"2024-06-03T01:04:48.749134+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:26:16.150699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.266677+0000","last_change":"2024-06-03T01:04:54.365914+0000","last_active":"2024-06-03T01:08:04.266677+0000","last_peered":"2024-06-03T01:08:04.266677+0000","last_clean":"2024-06-03T01:08:04.266677+0000","last_became_active":"2024-06-03T01:04:54.365507+0000","last_became_peered":"2024-06-03T01:04:54.365507+0000","last_unstale":"2024-06-03T01:08:04.266677+0000","last_undegraded":"2024-06-03T01:08:04.266677+0000","last_fullsized":"2024-06-03T01:08:04.266677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:24:04.278102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.260307+0000","last_change":"2024-06-03T01:04:54.367546+0000","last_active":"2024-06-03T01:08:04.260307+0000","last_peered":"2024-06-03T01:08:04.260307+0000","last_clean":"2024-06-03T01:08:04.260307+0000","last_became_active":"2024-06-03T01:04:54.367116+0000","last_became_peered":"2024-06-03T01:04:54.367116+0000","last_unstale":"2024-06-03T01:08:04.260307+0000","last_undegraded":"2024-06-03T01:08:04.260307+0000","last_fullsized":"2024-06-03T01:08:04.260307+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:52:15.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.255857+0000","last_change":"2024-06-03T01:04:54.366779+0000","last_active":"2024-06-03T01:08:04.255857+0000","last_peered":"2024-06-03T01:08:04.255857+0000","last_clean":"2024-06-03T01:08:04.255857+0000","last_became_active":"2024-06-03T01:04:54.366420+0000","last_became_peered":"2024-06-03T01:04:54.366420+0000","last_unstale":"2024-06-03T01:08:04.255857+0000","last_undegraded":"2024-06-03T01:08:04.255857+0000","last_fullsized":"2024-06-03T01:08:04.255857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:31:16.277064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.261192+0000","last_change":"2024-06-03T01:04:54.393396+0000","last_active":"2024-06-03T01:08:04.261192+0000","last_peered":"2024-06-03T01:08:04.261192+0000","last_clean":"2024-06-03T01:08:04.261192+0000","last_became_active":"2024-06-03T01:04:54.393071+0000","last_became_peered":"2024-06-03T01:04:54.393071+0000","last_unstale":"2024-06-03T01:08:04.261192+0000","last_undegraded":"2024-06-03T01:08:04.261192+0000","last_fullsized":"2024-06-03T01:08:04.261192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:50:25.507494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.290933+0000","last_change":"2024-06-03T01:05:35.262012+0000","last_active":"2024-06-03T01:08:04.290933+0000","last_peered":"2024-06-03T01:08:04.290933+0000","last_clean":"2024-06-03T01:08:04.290933+0000","last_became_active":"2024-06-03T01:05:35.261644+0000","last_became_peered":"2024-06-03T01:05:35.261644+0000","last_unstale":"2024-06-03T01:08:04.290933+0000","last_undegraded":"2024-06-03T01:08:04.290933+0000","last_fullsized":"2024-06-03T01:08:04.290933+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:33:33.480127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.288883+0000","last_change":"2024-06-03T01:04:54.388884+0000","last_active":"2024-06-03T01:08:04.288883+0000","last_peered":"2024-06-03T01:08:04.288883+0000","last_clean":"2024-06-03T01:08:04.288883+0000","last_became_active":"2024-06-03T01:04:54.387969+0000","last_became_peered":"2024-06-03T01:04:54.387969+0000","last_unstale":"2024-06-03T01:08:04.288883+0000","last_undegraded":"2024-06-03T01:08:04.288883+0000","last_fullsized":"2024-06-03T01:08:04.288883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:29:13.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.254966+0000","last_change":"2024-06-03T01:04:54.364412+0000","last_active":"2024-06-03T01:08:04.254966+0000","last_peered":"2024-06-03T01:08:04.254966+0000","last_clean":"2024-06-03T01:08:04.254966+0000","last_became_active":"2024-06-03T01:04:54.364053+0000","last_became_peered":"2024-06-03T01:04:54.364053+0000","last_unstale":"2024-06-03T01:08:04.254966+0000","last_undegraded":"2024-06-03T01:08:04.254966+0000","last_fullsized":"2024-06-03T01:08:04.254966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:47:26.760514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.257347+0000","last_change":"2024-06-03T01:04:54.370604+0000","last_active":"2024-06-03T01:08:04.257347+0000","last_peered":"2024-06-03T01:08:04.257347+0000","last_clean":"2024-06-03T01:08:04.257347+0000","last_became_active":"2024-06-03T01:04:54.370288+0000","last_became_peered":"2024-06-03T01:04:54.370288+0000","last_unstale":"2024-06-03T01:08:04.257347+0000","last_undegraded":"2024-06-03T01:08:04.257347+0000","last_fullsized":"2024-06-03T01:08:04.257347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:34:49.350481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.274458+0000","last_change":"2024-06-03T01:05:35.260161+0000","last_active":"2024-06-03T01:08:04.274458+0000","last_peered":"2024-06-03T01:08:04.274458+0000","last_clean":"2024-06-03T01:08:04.274458+0000","last_became_active":"2024-06-03T01:05:35.259725+0000","last_became_peered":"2024-06-03T01:05:35.259725+0000","last_unstale":"2024-06-03T01:08:04.274458+0000","last_undegraded":"2024-06-03T01:08:04.274458+0000","last_fullsized":"2024-06-03T01:08:04.274458+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:53:36.698726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.275477+0000","last_change":"2024-06-03T01:04:54.391161+0000","last_active":"2024-06-03T01:08:04.275477+0000","last_peered":"2024-06-03T01:08:04.275477+0000","last_clean":"2024-06-03T01:08:04.275477+0000","last_became_active":"2024-06-03T01:04:54.390789+0000","last_became_peered":"2024-06-03T01:04:54.390789+0000","last_unstale":"2024-06-03T01:08:04.275477+0000","last_undegraded":"2024-06-03T01:08:04.275477+0000","last_fullsized":"2024-06-03T01:08:04.275477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:54:07.520300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":24,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.280657+0000","last_change":"2024-06-03T01:05:35.257377+0000","last_active":"2024-06-03T01:08:04.280657+0000","last_peered":"2024-06-03T01:08:04.280657+0000","last_clean":"2024-06-03T01:08:04.280657+0000","last_became_active":"2024-06-03T01:05:35.256864+0000","last_became_peered":"2024-06-03T01:05:35.256864+0000","last_unstale":"2024-06-03T01:08:04.280657+0000","last_undegraded":"2024-06-03T01:08:04.280657+0000","last_fullsized":"2024-06-03T01:08:04.280657+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:06:16.396763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.271939+0000","last_change":"2024-06-03T01:04:54.385235+0000","last_active":"2024-06-03T01:08:04.271939+0000","last_peered":"2024-06-03T01:08:04.271939+0000","last_clean":"2024-06-03T01:08:04.271939+0000","last_became_active":"2024-06-03T01:04:54.384834+0000","last_became_peered":"2024-06-03T01:04:54.384834+0000","last_unstale":"2024-06-03T01:08:04.271939+0000","last_undegraded":"2024-06-03T01:08:04.271939+0000","last_fullsized":"2024-06-03T01:08:04.271939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:10:49.586108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.292093+0000","last_change":"2024-06-03T01:04:54.389627+0000","last_active":"2024-06-03T01:08:04.292093+0000","last_peered":"2024-06-03T01:08:04.292093+0000","last_clean":"2024-06-03T01:08:04.292093+0000","last_became_active":"2024-06-03T01:04:54.388430+0000","last_became_peered":"2024-06-03T01:04:54.388430+0000","last_unstale":"2024-06-03T01:08:04.292093+0000","last_undegraded":"2024-06-03T01:08:04.292093+0000","last_fullsized":"2024-06-03T01:08:04.292093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:32:44.376952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-03T01:08:04.254013+0000","last_change":"2024-06-03T01:04:54.366394+0000","last_active":"2024-06-03T01:08:04.254013+0000","last_peered":"2024-06-03T01:08:04.254013+0000","last_clean":"2024-06-03T01:08:04.254013+0000","last_became_active":"2024-06-03T01:04:54.366067+0000","last_became_peered":"2024-06-03T01:04:54.366067+0000","last_unstale":"2024-06-03T01:08:04.254013+0000","last_undegraded":"2024-06-03T01:08:04.254013+0000","last_fullsized":"2024-06-03T01:08:04.254013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-03T01:04:53.333884+0000","last_clean_scrub_stamp":"2024-06-03T01:04:53.333884+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:18:03.772654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76846,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1301,"num_read_kb":734,"num_write":266,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":230538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351036,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":575494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612673,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37152,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708000,"statfs":{"total":95995035648,"available":95956992000,"internally_reserved":0,"allocated":1859584,"data_stored":1230498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776200,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1187840,"data_stored":578877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037838,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":575047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299475,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1777664,"data_stored":1172327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463001,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32308,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712844,"statfs":{"total":95995035648,"available":95961952256,"internally_reserved":0,"allocated":1159168,"data_stored":570816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1232896,"data_stored":640677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":572272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28240,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716912,"statfs":{"total":95995035648,"available":95966117888,"internally_reserved":0,"allocated":1253376,"data_stored":641428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1171456,"data_stored":569993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920868,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32316,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712836,"statfs":{"total":95995035648,"available":95961944064,"internally_reserved":0,"allocated":1167360,"data_stored":574359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724638,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1785856,"data_stored":1167378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-03T01:08:28.677 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-03T01:08:28.677 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-03T01:08:28.677 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-03T01:08:28.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-03T01:08:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:28 smithi107 ceph-mon[36622]: from='client.24926 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:08:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:28 smithi107 ceph-mon[36622]: pgmap v447: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:28.788 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:28 smithi094 ceph-mon[32119]: from='client.24926 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:08:28.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:28 smithi094 ceph-mon[32119]: pgmap v447: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:28 smithi179 ceph-mon[37041]: from='client.24926 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-03T01:08:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:28 smithi179 ceph-mon[37041]: pgmap v447: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:29.166 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:30 smithi094 ceph-mon[32119]: pgmap v448: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:30.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:30 smithi107 ceph-mon[36622]: pgmap v448: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:30 smithi179 ceph-mon[37041]: pgmap v448: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 -- 172.21.15.94:0/3925349976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 msgr2=0x7fb3dc111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/3925349976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 0x7fb3dc111280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400b0a0 tx=0x7fb3d400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 -- 172.21.15.94:0/3925349976 shutdown_connections 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/3925349976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 0x7fb3dc111280 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/3925349976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc0746c0 0x7fb3dc074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/3925349976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 0x7fb3dc10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 -- 172.21.15.94:0/3925349976 >> 172.21.15.94:0/3925349976 conn(0x7fb3dc0795f0 msgr2=0x7fb3dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:31.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.454+0000 7fb3e1e00700 1 -- 172.21.15.94:0/3925349976 shutdown_connections 2024-06-03T01:08:31.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.455+0000 7fb3e1e00700 1 -- 172.21.15.94:0/3925349976 wait complete. 2024-06-03T01:08:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 Processor -- start 2024-06-03T01:08:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 -- start start 2024-06-03T01:08:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc0746c0 0x7fb3dc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 0x7fb3dc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:31.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 0x7fb3dc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc1149d0 con 0x7fb3dc0746c0 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc114850 con 0x7fb3dc10ee90 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.456+0000 7fb3e1e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc114b50 con 0x7fb3dc074fe0 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3daa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc0746c0 0x7fb3dc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 0x7fb3dc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 0x7fb3dc19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53428/0 (socket says 172.21.15.94:53428) 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 -- 172.21.15.94:0/2264540174 learned_addr learned my addr 172.21.15.94:0/2264540174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3db400700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 0x7fb3dc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 -- 172.21.15.94:0/2264540174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 msgr2=0x7fb3dc19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 0x7fb3dc19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 -- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc0746c0 msgr2=0x7fb3dc19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc0746c0 0x7fb3dc19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 -- 172.21.15.94:0/2264540174 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc19fe00 con 0x7fb3dc074fe0 2024-06-03T01:08:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3db400700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 0x7fb3dc19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:08:31.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.457+0000 7fb3d3e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 0x7fb3dc19b380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9c0 tx=0x7fb3c400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.458+0000 7fb3e0a00700 1 -- 172.21.15.94:0/2264540174 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400cbd0 con 0x7fb3dc074fe0 2024-06-03T01:08:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.458+0000 7fb3e0a00700 1 -- 172.21.15.94:0/2264540174 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c400cd70 con 0x7fb3dc074fe0 2024-06-03T01:08:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.458+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc1a00f0 con 0x7fb3dc074fe0 2024-06-03T01:08:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.458+0000 7fb3e0a00700 1 -- 172.21.15.94:0/2264540174 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4010430 con 0x7fb3dc074fe0 2024-06-03T01:08:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.458+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc1a7970 con 0x7fb3dc074fe0 2024-06-03T01:08:31.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.459+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3dc069180 con 0x7fb3dc074fe0 2024-06-03T01:08:31.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.461+0000 7fb3e0a00700 1 -- 172.21.15.94:0/2264540174 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb3c4004750 con 0x7fb3dc074fe0 2024-06-03T01:08:31.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.462+0000 7fb3e0a00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3c00a2c20 0x7fb3c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:31.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.463+0000 7fb3e0a00700 1 -- 172.21.15.94:0/2264540174 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb3c40cf880 con 0x7fb3dc074fe0 2024-06-03T01:08:31.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.463+0000 7fb3daa00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3c00a2c20 0x7fb3c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:31.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.463+0000 7fb3daa00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3c00a2c20 0x7fb3c00a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009c60 tx=0x7fb3cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:31.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.465+0000 7fb3e0a00700 1 -- 172.21.15.94:0/2264540174 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3c4053850 con 0x7fb3dc074fe0 2024-06-03T01:08:31.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.735+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb3dc19c150 con 0x7fb3dc074fe0 2024-06-03T01:08:31.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.736+0000 7fb3e0a00700 1 -- 172.21.15.94:0/2264540174 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb3dc19c150 con 0x7fb3dc074fe0 2024-06-03T01:08:31.737 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:08:31.737 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-03T01:08:31.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3c00a2c20 msgr2=0x7fb3c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:31.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3c00a2c20 0x7fb3c00a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009c60 tx=0x7fb3cc009260 comp rx=0 tx=0).stop 2024-06-03T01:08:31.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 msgr2=0x7fb3dc19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:31.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 0x7fb3dc19b380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9c0 tx=0x7fb3c400ecd0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 shutdown_connections 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3dc10ee90 0x7fb3dc19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb3c00a2c20 0x7fb3c00a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3dc0746c0 0x7fb3dc19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 --2- 172.21.15.94:0/2264540174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3dc074fe0 0x7fb3dc19b380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.740+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 >> 172.21.15.94:0/2264540174 conn(0x7fb3dc0795f0 msgr2=0x7fb3dc0ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.741+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 shutdown_connections 2024-06-03T01:08:31.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:31.741+0000 7fb3e1e00700 1 -- 172.21.15.94:0/2264540174 wait complete. 2024-06-03T01:08:32.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:31 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2264540174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T01:08:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:31 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2264540174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T01:08:32.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:31 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2264540174' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-03T01:08:32.641 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-03T01:08:32.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:08:32.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:32 smithi094 ceph-mon[32119]: pgmap v449: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:33.086 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:33.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:32 smithi179 ceph-mon[37041]: pgmap v449: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:33.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:32 smithi107 ceph-mon[36622]: pgmap v449: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:34 smithi107 ceph-mon[36622]: pgmap v450: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:34.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:34 smithi094 ceph-mon[32119]: pgmap v450: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:34.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:34 smithi179 ceph-mon[37041]: pgmap v450: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:35.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.522+0000 7f2513800700 1 -- 172.21.15.94:0/2956148704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 msgr2=0x7f250c074ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:35.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.522+0000 7f2513800700 1 --2- 172.21.15.94:0/2956148704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 0x7f250c074ae0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2500009990 tx=0x7f2500009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.523+0000 7f2513800700 1 -- 172.21.15.94:0/2956148704 shutdown_connections 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.523+0000 7f2513800700 1 --2- 172.21.15.94:0/2956148704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 0x7f250c074ae0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.523+0000 7f2513800700 1 --2- 172.21.15.94:0/2956148704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c10ee90 0x7f250c111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.523+0000 7f2513800700 1 --2- 172.21.15.94:0/2956148704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c075020 0x7f250c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.523+0000 7f2513800700 1 -- 172.21.15.94:0/2956148704 >> 172.21.15.94:0/2956148704 conn(0x7f250c079680 msgr2=0x7f250c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.523+0000 7f2513800700 1 -- 172.21.15.94:0/2956148704 shutdown_connections 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.523+0000 7f2513800700 1 -- 172.21.15.94:0/2956148704 wait complete. 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.524+0000 7f2513800700 1 Processor -- start 2024-06-03T01:08:35.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.524+0000 7f2513800700 1 -- start start 2024-06-03T01:08:35.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2513800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 0x7f250c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:35.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2513800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c075020 0x7f250c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:35.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2513800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c10ee90 0x7f250c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:35.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2513800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f250c114a50 con 0x7f250c10ee90 2024-06-03T01:08:35.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2513800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1148d0 con 0x7f250c074700 2024-06-03T01:08:35.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2513800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f250c114bd0 con 0x7f250c075020 2024-06-03T01:08:35.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2510e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 0x7f250c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f250be00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c075020 0x7f250c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2510e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 0x7f250c19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37558/0 (socket says 172.21.15.94:37558) 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2510e00700 1 -- 172.21.15.94:0/781416478 learned_addr learned my addr 172.21.15.94:0/781416478 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.525+0000 7f2511800700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c10ee90 0x7f250c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.526+0000 7f250be00700 1 -- 172.21.15.94:0/781416478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 msgr2=0x7f250c19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.526+0000 7f250be00700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 0x7f250c19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.526+0000 7f250be00700 1 -- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c10ee90 msgr2=0x7f250c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.526+0000 7f250be00700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c10ee90 0x7f250c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.526+0000 7f250be00700 1 -- 172.21.15.94:0/781416478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c19fef0 con 0x7f250c075020 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.526+0000 7f250be00700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c075020 0x7f250c19b420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e9f0 tx=0x7f24f800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.526+0000 7f2509600700 1 -- 172.21.15.94:0/781416478 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f800cb70 con 0x7f250c075020 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.527+0000 7f2509600700 1 -- 172.21.15.94:0/781416478 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f800ee90 con 0x7f250c075020 2024-06-03T01:08:35.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.527+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c1a01e0 con 0x7f250c075020 2024-06-03T01:08:35.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.527+0000 7f2509600700 1 -- 172.21.15.94:0/781416478 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8003dc0 con 0x7f250c075020 2024-06-03T01:08:35.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.527+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c1a7a60 con 0x7f250c075020 2024-06-03T01:08:35.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.528+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c069180 con 0x7f250c075020 2024-06-03T01:08:35.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.530+0000 7f2509600700 1 -- 172.21.15.94:0/781416478 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24f8004000 con 0x7f250c075020 2024-06-03T01:08:35.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.531+0000 7f2509600700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24f40a2ca0 0x7f24f40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:35.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.532+0000 7f2509600700 1 -- 172.21.15.94:0/781416478 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f24f80cf570 con 0x7f250c075020 2024-06-03T01:08:35.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.532+0000 7f2510e00700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24f40a2ca0 0x7f24f40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:35.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.532+0000 7f2510e00700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24f40a2ca0 0x7f24f40a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2500009750 tx=0x7f250000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:35.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.533+0000 7f2509600700 1 -- 172.21.15.94:0/781416478 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f24f809c180 con 0x7f250c075020 2024-06-03T01:08:35.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.806+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f250c19c270 con 0x7f250c075020 2024-06-03T01:08:35.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.807+0000 7f2509600700 1 -- 172.21.15.94:0/781416478 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f250c19c270 con 0x7f250c075020 2024-06-03T01:08:35.808 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:08:35.808 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:06:22.586009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24478,24485,24490,24505,24524]},"id":1}]} 2024-06-03T01:08:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24f40a2ca0 msgr2=0x7f24f40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24f40a2ca0 0x7f24f40a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2500009750 tx=0x7f250000b280 comp rx=0 tx=0).stop 2024-06-03T01:08:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c075020 msgr2=0x7f250c19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:35.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c075020 0x7f250c19b420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e9f0 tx=0x7f24f800ed00 comp rx=0 tx=0).stop 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 shutdown_connections 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f250c074700 0x7f250c19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24f40a2ca0 0x7f24f40a5160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f250c10ee90 0x7f250c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 --2- 172.21.15.94:0/781416478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f250c075020 0x7f250c19b420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 >> 172.21.15.94:0/781416478 conn(0x7f250c079680 msgr2=0x7f250c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 shutdown_connections 2024-06-03T01:08:35.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:35.811+0000 7f2513800700 1 -- 172.21.15.94:0/781416478 wait complete. 2024-06-03T01:08:35.813 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-06-03T01:08:36.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:35 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/781416478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:35 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/781416478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:35 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/781416478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:36.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:08:37.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:36 smithi094 ceph-mon[32119]: pgmap v451: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:37.270 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:37.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:36 smithi107 ceph-mon[36622]: pgmap v451: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:37.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:36 smithi179 ceph-mon[37041]: pgmap v451: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:38.669 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:38 smithi094 ceph-mon[32119]: pgmap v452: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:38 smithi107 ceph-mon[36622]: pgmap v452: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:38 smithi179 ceph-mon[37041]: pgmap v452: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 -- 172.21.15.94:0/3219409676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8107400 msgr2=0x7fdab81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 --2- 172.21.15.94:0/3219409676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8107400 0x7fdab81077e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b0a0 tx=0x7fdaa800b3b0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 -- 172.21.15.94:0/3219409676 shutdown_connections 2024-06-03T01:08:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 --2- 172.21.15.94:0/3219409676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab8101d50 0x7fdab810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 --2- 172.21.15.94:0/3219409676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8107400 0x7fdab81077e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 --2- 172.21.15.94:0/3219409676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab8101390 0x7fdab8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 -- 172.21.15.94:0/3219409676 >> 172.21.15.94:0/3219409676 conn(0x7fdab8078340 msgr2=0x7fdab8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:39.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.725+0000 7fdabe000700 1 -- 172.21.15.94:0/3219409676 shutdown_connections 2024-06-03T01:08:39.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.726+0000 7fdabe000700 1 -- 172.21.15.94:0/3219409676 wait complete. 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 Processor -- start 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 -- start start 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8101390 0x7fdab819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab8101d50 0x7fdab819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab8107400 0x7fdab8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8111670 con 0x7fdab8101390 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81114f0 con 0x7fdab8101d50 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdabe000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81117f0 con 0x7fdab8107400 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdab7400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8101390 0x7fdab819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdab7400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8101390 0x7fdab819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39592/0 (socket says 172.21.15.94:39592) 2024-06-03T01:08:39.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.727+0000 7fdab7400700 1 -- 172.21.15.94:0/2052423810 learned_addr learned my addr 172.21.15.94:0/2052423810 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:39.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdaa6a00700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab8101d50 0x7fdab819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:39.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdab7e00700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab8107400 0x7fdab8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:39.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdab7400700 1 -- 172.21.15.94:0/2052423810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab8107400 msgr2=0x7fdab8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:39.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdab7400700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab8107400 0x7fdab8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdab7400700 1 -- 172.21.15.94:0/2052423810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab8101d50 msgr2=0x7fdab819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:39.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdab7400700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab8101d50 0x7fdab819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdab7400700 1 -- 172.21.15.94:0/2052423810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab81991e0 con 0x7fdab8101390 2024-06-03T01:08:39.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.728+0000 7fdab7400700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8101390 0x7fdab819e870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80062a0 tx=0x7fdaa8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:39.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.729+0000 7fdab4c00700 1 -- 172.21.15.94:0/2052423810 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8021070 con 0x7fdab8101390 2024-06-03T01:08:39.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.729+0000 7fdab4c00700 1 -- 172.21.15.94:0/2052423810 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa8007b80 con 0x7fdab8101390 2024-06-03T01:08:39.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.729+0000 7fdab4c00700 1 -- 172.21.15.94:0/2052423810 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80052a0 con 0x7fdab8101390 2024-06-03T01:08:39.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.729+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab8199470 con 0x7fdab8101390 2024-06-03T01:08:39.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.729+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab81a5740 con 0x7fdab8101390 2024-06-03T01:08:39.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.730+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab8069180 con 0x7fdab8101390 2024-06-03T01:08:39.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.732+0000 7fdab4c00700 1 -- 172.21.15.94:0/2052423810 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdaa8024070 con 0x7fdab8101390 2024-06-03T01:08:39.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.734+0000 7fdab4c00700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda980a2c20 0x7fda980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:39.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.735+0000 7fdab4c00700 1 -- 172.21.15.94:0/2052423810 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fdaa800f020 con 0x7fdab8101390 2024-06-03T01:08:39.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.735+0000 7fdaa6a00700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda980a2c20 0x7fda980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:39.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.736+0000 7fdaa6a00700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda980a2c20 0x7fda980a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdab80788d0 tx=0x7fdaa00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:39.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.738+0000 7fdab4c00700 1 -- 172.21.15.94:0/2052423810 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdaa802b020 con 0x7fdab8101390 2024-06-03T01:08:39.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.994+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab81a5910 con 0x7fdab8101390 2024-06-03T01:08:39.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.996+0000 7fdab4c00700 1 -- 172.21.15.94:0/2052423810 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7fdaa809da60 con 0x7fdab8101390 2024-06-03T01:08:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:08:39.996 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:06:22.586009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24478,24485,24490,24505,24524]},"id":1}]} 2024-06-03T01:08:39.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda980a2c20 msgr2=0x7fda980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:39.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda980a2c20 0x7fda980a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdab80788d0 tx=0x7fdaa00073d0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8101390 msgr2=0x7fdab819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8101390 0x7fdab819e870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80062a0 tx=0x7fdaa8004d70 comp rx=0 tx=0).stop 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 shutdown_connections 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fda980a2c20 0x7fda980a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdab8107400 0x7fdab8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdab8101390 0x7fdab819e870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 --2- 172.21.15.94:0/2052423810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdab8101d50 0x7fdab819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.998+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 >> 172.21.15.94:0/2052423810 conn(0x7fdab8078340 msgr2=0x7fdab8100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:39.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.999+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 shutdown_connections 2024-06-03T01:08:40.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:39.999+0000 7fdabe000700 1 -- 172.21.15.94:0/2052423810 wait complete. 2024-06-03T01:08:40.000 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-06-03T01:08:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:40 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2052423810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:40 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2052423810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:40 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2052423810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:41.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:08:41.104 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:41 smithi094 ceph-mon[32119]: pgmap v453: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:41.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:41 smithi179 ceph-mon[37041]: pgmap v453: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:41.477 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:41.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:41 smithi107 ceph-mon[36622]: pgmap v453: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:42 smithi107 ceph-mon[36622]: pgmap v454: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:42.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:42 smithi094 ceph-mon[32119]: pgmap v454: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:42.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:42 smithi179 ceph-mon[37041]: pgmap v454: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 -- 172.21.15.94:0/1148435926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34075460 msgr2=0x7f3c3410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 --2- 172.21.15.94:0/1148435926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34075460 0x7f3c3410e690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009990 tx=0x7f3c1c009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 -- 172.21.15.94:0/1148435926 shutdown_connections 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 --2- 172.21.15.94:0/1148435926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 0x7f3c34111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 --2- 172.21.15.94:0/1148435926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c34074ab0 0x7f3c34074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 --2- 172.21.15.94:0/1148435926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34075460 0x7f3c3410e690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 -- 172.21.15.94:0/1148435926 >> 172.21.15.94:0/1148435926 conn(0x7f3c340fde80 msgr2=0x7f3c341002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 -- 172.21.15.94:0/1148435926 shutdown_connections 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.931+0000 7f3c33e00700 1 -- 172.21.15.94:0/1148435926 wait complete. 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.932+0000 7f3c33e00700 1 Processor -- start 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.932+0000 7f3c33e00700 1 -- start start 2024-06-03T01:08:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34074ab0 0x7f3c3419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c34075460 0x7f3c3419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 0x7f3c3419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c341149f0 con 0x7f3c34074ab0 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34114870 con 0x7f3c3410ed00 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34114b70 con 0x7f3c34075460 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 0x7f3c3419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c32a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34074ab0 0x7f3c3419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 0x7f3c3419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37602/0 (socket says 172.21.15.94:37602) 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33400700 1 -- 172.21.15.94:0/2951438593 learned_addr learned my addr 172.21.15.94:0/2951438593 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c2a000700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c34075460 0x7f3c3419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33400700 1 -- 172.21.15.94:0/2951438593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c34075460 msgr2=0x7f3c3419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.933+0000 7f3c33400700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c34075460 0x7f3c3419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c33400700 1 -- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34074ab0 msgr2=0x7f3c3419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c33400700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34074ab0 0x7f3c3419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c33400700 1 -- 172.21.15.94:0/2951438593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3419fe10 con 0x7f3c3410ed00 2024-06-03T01:08:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c33400700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 0x7f3c3419f6d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00ef30 tx=0x7f3c2c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c2be00700 1 -- 172.21.15.94:0/2951438593 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0092e0 con 0x7f3c3410ed00 2024-06-03T01:08:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c341a00a0 con 0x7f3c3410ed00 2024-06-03T01:08:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c2be00700 1 -- 172.21.15.94:0/2951438593 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c004bb0 con 0x7f3c3410ed00 2024-06-03T01:08:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c2be00700 1 -- 172.21.15.94:0/2951438593 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c004280 con 0x7f3c3410ed00 2024-06-03T01:08:43.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.934+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c341a7980 con 0x7f3c3410ed00 2024-06-03T01:08:43.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.935+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c34069180 con 0x7f3c3410ed00 2024-06-03T01:08:43.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.938+0000 7f3c2be00700 1 -- 172.21.15.94:0/2951438593 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c2c044d10 con 0x7f3c3410ed00 2024-06-03T01:08:43.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.939+0000 7f3c2be00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f3c180a2c20 0x7f3c180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:43.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.939+0000 7f3c2be00700 1 -- 172.21.15.94:0/2951438593 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f3c2c0cf2b0 con 0x7f3c3410ed00 2024-06-03T01:08:43.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.939+0000 7f3c32a00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f3c180a2c20 0x7f3c180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:43.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.940+0000 7f3c32a00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f3c180a2c20 0x7f3c180a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400ac50 tx=0x7f3c2400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:43.942+0000 7f3c2be00700 1 -- 172.21.15.94:0/2951438593 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c2c022020 con 0x7f3c3410ed00 2024-06-03T01:08:44.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.170+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c341a7b50 con 0x7f3c3410ed00 2024-06-03T01:08:44.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.172+0000 7f3c2be00700 1 -- 172.21.15.94:0/2951438593 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24723 (secure 0 0 0) 0x7f3c2c010040 con 0x7f3c3410ed00 2024-06-03T01:08:44.174 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:08:44.174 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":100,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:06:35.341979+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.175+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f3c180a2c20 msgr2=0x7f3c180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.175+0000 7f3c33e00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f3c180a2c20 0x7f3c180a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400ac50 tx=0x7f3c2400a380 comp rx=0 tx=0).stop 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.175+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 msgr2=0x7f3c3419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.175+0000 7f3c33e00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 0x7f3c3419f6d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00ef30 tx=0x7f3c2c00c3a0 comp rx=0 tx=0).stop 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 shutdown_connections 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c3410ed00 0x7f3c3419f6d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f3c180a2c20 0x7f3c180a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:44.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c34074ab0 0x7f3c3419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 --2- 172.21.15.94:0/2951438593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c34075460 0x7f3c3419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 >> 172.21.15.94:0/2951438593 conn(0x7f3c340fde80 msgr2=0x7f3c34110f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 shutdown_connections 2024-06-03T01:08:44.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:44.176+0000 7f3c33e00700 1 -- 172.21.15.94:0/2951438593 wait complete. 2024-06-03T01:08:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:44 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2951438593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:08:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:44 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2951438593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:08:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:44 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2951438593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:08:45.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:08:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:45 smithi107 ceph-mon[36622]: pgmap v455: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:45 smithi094 ceph-mon[32119]: pgmap v455: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:45 smithi179 ceph-mon[37041]: pgmap v455: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:45.656 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:46.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:46 smithi094 ceph-mon[32119]: pgmap v456: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:46 smithi107 ceph-mon[36622]: pgmap v456: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:46.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:46 smithi179 ceph-mon[37041]: pgmap v456: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:48.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.124+0000 7f1bef800700 1 -- 172.21.15.94:0/3969332924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101390 msgr2=0x7f1be8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:48.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.124+0000 7f1bef800700 1 --2- 172.21.15.94:0/3969332924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101390 0x7f1be8101810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009990 tx=0x7f1bd4009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.124+0000 7f1bef800700 1 -- 172.21.15.94:0/3969332924 shutdown_connections 2024-06-03T01:08:48.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.124+0000 7f1bef800700 1 --2- 172.21.15.94:0/3969332924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be8101d50 0x7f1be810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.124+0000 7f1bef800700 1 --2- 172.21.15.94:0/3969332924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be8107400 0x7f1be81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.124+0000 7f1bef800700 1 --2- 172.21.15.94:0/3969332924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101390 0x7f1be8101810 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.124+0000 7f1bef800700 1 -- 172.21.15.94:0/3969332924 >> 172.21.15.94:0/3969332924 conn(0x7f1be8078340 msgr2=0x7f1be8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:48.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.125+0000 7f1bef800700 1 -- 172.21.15.94:0/3969332924 shutdown_connections 2024-06-03T01:08:48.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.125+0000 7f1bef800700 1 -- 172.21.15.94:0/3969332924 wait complete. 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 Processor -- start 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 -- start start 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be8101390 0x7f1be819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101d50 0x7f1be819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be8107400 0x7f1be819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1be81116a0 con 0x7f1be8101390 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8111520 con 0x7f1be8101d50 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.126+0000 7f1bef800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8111820 con 0x7f1be8107400 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bece00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be8101390 0x7f1be819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:48.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1be7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101d50 0x7f1be819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:48.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bed800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be8107400 0x7f1be819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:48.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bece00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be8101390 0x7f1be819ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33578/0 (socket says 172.21.15.94:33578) 2024-06-03T01:08:48.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bece00700 1 -- 172.21.15.94:0/3493343422 learned_addr learned my addr 172.21.15.94:0/3493343422 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:48.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bed800700 1 -- 172.21.15.94:0/3493343422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101d50 msgr2=0x7f1be819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bed800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101d50 0x7f1be819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bed800700 1 -- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be8101390 msgr2=0x7f1be819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bed800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be8101390 0x7f1be819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.127+0000 7f1bed800700 1 -- 172.21.15.94:0/3493343422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be819fe40 con 0x7f1be8107400 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.128+0000 7f1bed800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be8107400 0x7f1be819f700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be000efc0 tx=0x7f1be000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.128+0000 7f1be5600700 1 -- 172.21.15.94:0/3493343422 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be00092e0 con 0x7f1be8107400 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.128+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be81a00d0 con 0x7f1be8107400 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.128+0000 7f1be5600700 1 -- 172.21.15.94:0/3493343422 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be0004bb0 con 0x7f1be8107400 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.128+0000 7f1be5600700 1 -- 172.21.15.94:0/3493343422 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0004280 con 0x7f1be8107400 2024-06-03T01:08:48.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.128+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be81a79b0 con 0x7f1be8107400 2024-06-03T01:08:48.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.129+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be8069180 con 0x7f1be8107400 2024-06-03T01:08:48.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.131+0000 7f1be5600700 1 -- 172.21.15.94:0/3493343422 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1be001b030 con 0x7f1be8107400 2024-06-03T01:08:48.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.133+0000 7f1be5600700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1bd00a2c20 0x7f1bd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:48.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.133+0000 7f1be5600700 1 -- 172.21.15.94:0/3493343422 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1be00cf900 con 0x7f1be8107400 2024-06-03T01:08:48.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.133+0000 7f1bece00700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1bd00a2c20 0x7f1bd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:48.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.134+0000 7f1bece00700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1bd00a2c20 0x7f1bd00a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009ca0 tx=0x7f1bdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:48.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.135+0000 7f1be5600700 1 -- 172.21.15.94:0/3493343422 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1be0022020 con 0x7f1be8107400 2024-06-03T01:08:48.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.404+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1be81a7b80 con 0x7f1be8107400 2024-06-03T01:08:48.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.405+0000 7f1be5600700 1 -- 172.21.15.94:0/3493343422 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f1be0010040 con 0x7f1be8107400 2024-06-03T01:08:48.407 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:08:48.407 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:06:22.586009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24478,24485,24490,24505,24524]},"id":1}]} 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1bd00a2c20 msgr2=0x7f1bd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1bd00a2c20 0x7f1bd00a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009ca0 tx=0x7f1bdc009260 comp rx=0 tx=0).stop 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be8107400 msgr2=0x7f1be819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be8107400 0x7f1be819f700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be000efc0 tx=0x7f1be000c3a0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 shutdown_connections 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1bd00a2c20 0x7f1bd00a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be8107400 0x7f1be819f700 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1be8101390 0x7f1be819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 --2- 172.21.15.94:0/3493343422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1be8101d50 0x7f1be819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:48.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 >> 172.21.15.94:0/3493343422 conn(0x7f1be8078340 msgr2=0x7f1be806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:48.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 shutdown_connections 2024-06-03T01:08:48.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:48.409+0000 7f1bef800700 1 -- 172.21.15.94:0/3493343422 wait complete. 2024-06-03T01:08:48.411 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-06-03T01:08:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:48 smithi107 ceph-mon[36622]: pgmap v457: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:48 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3493343422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:48 smithi094 ceph-mon[32119]: pgmap v457: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:48 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3493343422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:48 smithi179 ceph-mon[37041]: pgmap v457: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:48 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3493343422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:08:49.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-03T01:04:57.629349+0000', 'modified': '2024-06-03T01:06:22.586009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24505, 'mds_2': 24490, 'mds_3': 24485, 'mds_4': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14388': {'gid': 14388, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/848723917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 848723917}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 848723917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14400': {'gid': 14400, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6841/3437199777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6840', 'nonce': 3437199777}, {'type': 'v1', 'addr': '172.21.15.94:6841', 'nonce': 3437199777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6833/1330319563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 1330319563}, {'type': 'v1', 'addr': '172.21.15.107:6833', 'nonce': 1330319563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/2660538909', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 2660538909}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 2660538909}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/2528321857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 2528321857}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 2528321857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.179:6835/1407039746', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 1407039746}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 1407039746}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.107:6837/2537813051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 2537813051}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 2537813051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.179:6837/2233357781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 2233357781}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 2233357781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.179:6839/544081992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 544081992}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 544081992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.107:6839/1423733861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1423733861}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1423733861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24478, 24485, 24490, 24505, 24524]} 2024-06-03T01:08:49.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-03T01:08:49.266 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-03T01:08:49.734 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:50.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:50 smithi107 ceph-mon[36622]: pgmap v458: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:50.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:50 smithi094 ceph-mon[32119]: pgmap v458: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:50 smithi179 ceph-mon[37041]: pgmap v458: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:08:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 -- 172.21.15.94:0/1172977363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa33006ab60 msgr2=0x7fa33006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 --2- 172.21.15.94:0/1172977363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa33006ab60 0x7fa33006afe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0098f0 tx=0x7fa31c009c00 comp rx=0 tx=0).stop 2024-06-03T01:08:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 -- 172.21.15.94:0/1172977363 shutdown_connections 2024-06-03T01:08:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 --2- 172.21.15.94:0/1172977363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa330108520 0x7fa330108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 --2- 172.21.15.94:0/1172977363 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa33006b520 0x7fa33010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 --2- 172.21.15.94:0/1172977363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa33006ab60 0x7fa33006afe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 -- 172.21.15.94:0/1172977363 >> 172.21.15.94:0/1172977363 conn(0x7fa330078340 msgr2=0x7fa330078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.324+0000 7fa337600700 1 -- 172.21.15.94:0/1172977363 shutdown_connections 2024-06-03T01:08:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.325+0000 7fa337600700 1 -- 172.21.15.94:0/1172977363 wait complete. 2024-06-03T01:08:52.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.326+0000 7fa337600700 1 Processor -- start 2024-06-03T01:08:52.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.326+0000 7fa337600700 1 -- start start 2024-06-03T01:08:52.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.326+0000 7fa337600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa33006ab60 0x7fa330101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.326+0000 7fa337600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa33006b520 0x7fa330101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.326+0000 7fa337600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa330108520 0x7fa3301022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa337600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301127f0 con 0x7fa33006b520 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa337600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112670 con 0x7fa33006ab60 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa337600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa330112970 con 0x7fa330108520 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa32fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa33006b520 0x7fa330101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa33006ab60 0x7fa330101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa33006ab60 0x7fa330101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47202/0 (socket says 172.21.15.94:47202) 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa32fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa33006b520 0x7fa330101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33594/0 (socket says 172.21.15.94:33594) 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 -- 172.21.15.94:0/2946522769 learned_addr learned my addr 172.21.15.94:0/2946522769 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa335600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa330108520 0x7fa3301022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 -- 172.21.15.94:0/2946522769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa330108520 msgr2=0x7fa3301022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa330108520 0x7fa3301022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 -- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa33006b520 msgr2=0x7fa330101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa33006b520 0x7fa330101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.327+0000 7fa334c00700 1 -- 172.21.15.94:0/2946522769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3301a7240 con 0x7fa33006ab60 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.328+0000 7fa334c00700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa33006ab60 0x7fa330101840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9c0 tx=0x7fa32400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.328+0000 7fa32d600700 1 -- 172.21.15.94:0/2946522769 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32400cbd0 con 0x7fa33006ab60 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.328+0000 7fa32d600700 1 -- 172.21.15.94:0/2946522769 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa32400ee60 con 0x7fa33006ab60 2024-06-03T01:08:52.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.328+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3301a7530 con 0x7fa33006ab60 2024-06-03T01:08:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.328+0000 7fa32d600700 1 -- 172.21.15.94:0/2946522769 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324005630 con 0x7fa33006ab60 2024-06-03T01:08:52.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.328+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a79f0 con 0x7fa33006ab60 2024-06-03T01:08:52.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.333+0000 7fa32d600700 1 -- 172.21.15.94:0/2946522769 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa324004750 con 0x7fa33006ab60 2024-06-03T01:08:52.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.334+0000 7fa342067b80 1 Processor -- start 2024-06-03T01:08:52.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.334+0000 7fa342067b80 1 -- start start 2024-06-03T01:08:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.334+0000 7fa32d600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa3180a2c20 0x7fa3180a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa32d600700 1 -- 172.21.15.94:0/2946522769 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa3240cf640 con 0x7fa33006ab60 2024-06-03T01:08:52.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa342067b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa335600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa3180a2c20 0x7fa3180a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa342067b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa342067b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa342067b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559adb7ea9c0 con 0x559adb857df0 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa342067b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559adb82dab0 con 0x559adb8db560 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa342067b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559adb7e88a0 con 0x559adb8e62b0 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa334c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa334c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8db020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33604/0 (socket says 172.21.15.94:33604) 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa334c00700 1 -- 172.21.15.94:0/401943203 learned_addr learned my addr 172.21.15.94:0/401943203 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa32fe00700 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.335+0000 7fa335600700 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa335600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa3180a2c20 0x7fa3180a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa328009fd0 tx=0x7fa3280099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa334c00700 1 -- 172.21.15.94:0/401943203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 msgr2=0x559adb8e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa334c00700 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa334c00700 1 -- 172.21.15.94:0/401943203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 msgr2=0x559adb8db9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa334c00700 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8db9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa334c00700 1 -- 172.21.15.94:0/401943203 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559adb7d5f40 con 0x559adb857df0 2024-06-03T01:08:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa334c00700 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8db020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa32409ba50 tx=0x7fa32409b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283322cc49971f80 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:52.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa322000700 1 -- 172.21.15.94:0/401943203 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32409fe70 con 0x559adb857df0 2024-06-03T01:08:52.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.336+0000 7fa322000700 1 -- 172.21.15.94:0/401943203 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3240b4070 con 0x559adb857df0 2024-06-03T01:08:52.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa322000700 1 -- 172.21.15.94:0/401943203 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324016330 con 0x559adb857df0 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa342067b80 1 -- 172.21.15.94:0/401943203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 msgr2=0x559adb8db020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa342067b80 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8db020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa32409ba50 tx=0x7fa32409b510 comp rx=0 tx=0).stop 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa342067b80 1 -- 172.21.15.94:0/401943203 shutdown_connections 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa342067b80 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa342067b80 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8db020 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa342067b80 1 --2- 172.21.15.94:0/401943203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8db9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.337+0000 7fa342067b80 1 -- 172.21.15.94:0/401943203 >> 172.21.15.94:0/401943203 conn(0x559adb808120 msgr2=0x559adb8d7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:52.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.338+0000 7fa342067b80 1 -- 172.21.15.94:0/401943203 shutdown_connections 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.338+0000 7fa342067b80 1 -- 172.21.15.94:0/401943203 wait complete. 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.338+0000 7fa342067b80 1 -- start start 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 1 Processor -- start 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb8e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8e62b0 0x559adb8e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559adb7c9f10 con 0x559adb8e62b0 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559adb829a70 con 0x559adb8db560 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559adb8e8f10 con 0x559adb857df0 2024-06-03T01:08:52.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa342067b80 10 client.0 _ll_register_callbacks cb 0x559adb7c76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:08:52.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa32fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa335600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8e62b0 0x559adb8e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa32fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47222/0 (socket says 172.21.15.94:47222) 2024-06-03T01:08:52.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa32fe00700 1 -- 172.21.15.94:0/1827219594 learned_addr learned my addr 172.21.15.94:0/1827219594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:52.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.339+0000 7fa334c00700 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb8e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa32fe00700 1 -- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 msgr2=0x559adb8e59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa32fe00700 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb8e59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa32fe00700 1 -- 172.21.15.94:0/1827219594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8e62b0 msgr2=0x559adb8e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa32fe00700 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8e62b0 0x559adb8e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa32fe00700 1 -- 172.21.15.94:0/1827219594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559adb77c880 con 0x559adb8db560 2024-06-03T01:08:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa335600700 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8e62b0 0x559adb8e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa32fe00700 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e2ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0098f0 tx=0x7fa31c0094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f397c5a9140869da server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.340+0000 7fa316a00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.341+0000 7fa316a00700 1 -- 172.21.15.94:0/1827219594 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c005510 con 0x559adb8db560 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.341+0000 7fa316a00700 1 -- 172.21.15.94:0/1827219594 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa31c003680 con 0x559adb8db560 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.341+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3240b4070 con 0x559adb8db560 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.341+0000 7fa316a00700 1 -- 172.21.15.94:0/1827219594 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa31c02f050 con 0x559adb8db560 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.341+0000 7fa342067b80 10 client.24965 fetch_fsmap learned FSMap version 27 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.341+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559adb8f35f0 con 0x559adb8db560 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.341+0000 7fa316a00700 1 -- 172.21.15.94:0/1827219594 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3240b4070 con 0x559adb8db560 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa316a00700 1 -- 172.21.15.94:0/1827219594 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa31c02a030 con 0x559adb8db560 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa342067b80 10 client.24965 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa342067b80 10 client.24965 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T01:08:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa342067b80 20 client.24965 populate_metadata read hostname 'smithi094' 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa342067b80 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb8f5cc0 0x559adb8f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa342067b80 4 client.24965 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 --> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559adb83afb0 con 0x559adb8f5cc0 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.342+0000 7fa334c00700 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb8f5cc0 0x559adb8f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.343+0000 7fa334c00700 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb8f5cc0 0x559adb8f7130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f93118d8a7ec0b79 server_cookie=6752fed9d668a999 in_seq=0 out_seq=0 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.343+0000 7fa316a00700 10 client.24965 ms_handle_connect on v2:172.21.15.179:6834/1407039746 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa316a00700 1 -- 172.21.15.94:0/1827219594 <== mds.4 v2:172.21.15.179:6834/1407039746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559adb83afb0 con 0x559adb8f5cc0 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa316a00700 10 client.24965 handle_command_reply: tid=0 2024-06-03T01:08:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 client.24965 shutdown 2024-06-03T01:08:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 msgr2=0x559adb8e2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e2ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0098f0 tx=0x7fa31c0094c0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 shutdown_connections 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb8f5cc0 0x559adb8f7130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8e62b0 0x559adb8e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa316a00700 0 client.24965 ms_handle_reset on v2:172.21.15.179:6834/1407039746 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb8e59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 --2- 172.21.15.94:0/1827219594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e2ce0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 >> 172.21.15.94:0/1827219594 conn(0x559adb808120 msgr2=0x559adb8d66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.344+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 shutdown_connections 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.345+0000 7fa342067b80 1 -- 172.21.15.94:0/1827219594 wait complete. 2024-06-03T01:08:52.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.345+0000 7fa342067b80 20 client.24965 trim_cache size 0 max 16384 2024-06-03T01:08:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.368+0000 7fa342067b80 1 Processor -- start 2024-06-03T01:08:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.368+0000 7fa342067b80 1 -- start start 2024-06-03T01:08:52.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa342067b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa342067b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa342067b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa342067b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559adb8f5470 con 0x559adb857df0 2024-06-03T01:08:52.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa342067b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559adb8f5b40 con 0x559adb8db560 2024-06-03T01:08:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa342067b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559adb8f6210 con 0x559adb8e62b0 2024-06-03T01:08:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8e3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33618/0 (socket says 172.21.15.94:33618) 2024-06-03T01:08:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 -- 172.21.15.94:0/3198450512 learned_addr learned my addr 172.21.15.94:0/3198450512 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa335600700 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa32fe00700 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 -- 172.21.15.94:0/3198450512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 msgr2=0x559adb8e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 -- 172.21.15.94:0/3198450512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 msgr2=0x559adb8e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.369+0000 7fa334c00700 1 -- 172.21.15.94:0/3198450512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559adb8e1250 con 0x559adb857df0 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.370+0000 7fa334c00700 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8e3250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa32400f010 tx=0x7fa32409bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4ffc7e7b5419d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.370+0000 7fa321600700 1 -- 172.21.15.94:0/3198450512 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32409fe70 con 0x559adb857df0 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.370+0000 7fa321600700 1 -- 172.21.15.94:0/3198450512 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3240b4040 con 0x559adb857df0 2024-06-03T01:08:52.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.370+0000 7fa321600700 1 -- 172.21.15.94:0/3198450512 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32409bd20 con 0x559adb857df0 2024-06-03T01:08:52.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 -- 172.21.15.94:0/3198450512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 msgr2=0x559adb8e3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8e3250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa32400f010 tx=0x7fa32409bc40 comp rx=0 tx=0).stop 2024-06-03T01:08:52.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 -- 172.21.15.94:0/3198450512 shutdown_connections 2024-06-03T01:08:52.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb8e62b0 0x559adb8e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb857df0 0x559adb8e3250 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 --2- 172.21.15.94:0/3198450512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8db560 0x559adb8e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 -- 172.21.15.94:0/3198450512 >> 172.21.15.94:0/3198450512 conn(0x559adb808120 msgr2=0x559adb8e15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 -- 172.21.15.94:0/3198450512 shutdown_connections 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 -- 172.21.15.94:0/3198450512 wait complete. 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.371+0000 7fa342067b80 1 -- start start 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 1 Processor -- start 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb900ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8db560 0x559adb8fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8e62b0 0x559adb8fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559adb8f6fb0 con 0x559adb8db560 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559adb8f68e0 con 0x559adb8e62b0 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559adb8f7680 con 0x559adb857df0 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa342067b80 10 client.0 _ll_register_callbacks cb 0x559adb7c76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.372+0000 7fa334c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb900ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa32fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8db560 0x559adb8fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa334c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb900ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55504/0 (socket says 172.21.15.94:55504) 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa334c00700 1 -- 172.21.15.94:0/763197943 learned_addr learned my addr 172.21.15.94:0/763197943 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa335600700 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8e62b0 0x559adb8fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa32fe00700 1 -- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 msgr2=0x559adb900ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa32fe00700 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb900ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa32fe00700 1 -- 172.21.15.94:0/763197943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8e62b0 msgr2=0x559adb8fdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa32fe00700 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8e62b0 0x559adb8fdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa32fe00700 1 -- 172.21.15.94:0/763197943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559adb8fe310 con 0x559adb8db560 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa334c00700 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb900ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa32fe00700 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8db560 0x559adb8fd790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa31c000c00 tx=0x7fa31c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb5afad996ea9b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.373+0000 7fa316a00700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa316a00700 1 -- 172.21.15.94:0/763197943 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0058e0 con 0x559adb8db560 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3240b4040 con 0x559adb8db560 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa316a00700 1 -- 172.21.15.94:0/763197943 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa31c00b560 con 0x559adb8db560 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa316a00700 1 -- 172.21.15.94:0/763197943 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa31c042050 con 0x559adb8db560 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa342067b80 10 client.14856 fetch_fsmap learned FSMap version 27 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559adb961550 con 0x559adb8db560 2024-06-03T01:08:52.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa316a00700 1 -- 172.21.15.94:0/763197943 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3240b4040 con 0x559adb8db560 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.374+0000 7fa316a00700 1 -- 172.21.15.94:0/763197943 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa31c02a040 con 0x559adb8db560 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.375+0000 7fa342067b80 10 client.14856 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.375+0000 7fa342067b80 10 client.14856 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.375+0000 7fa342067b80 20 client.14856 populate_metadata read hostname 'smithi094' 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.375+0000 7fa342067b80 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb965f10 0x559adb964c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.375+0000 7fa342067b80 4 client.14856 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.375+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 --> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] -- command(tid 0: {"prefix": "status"}) -- 0x559adb835530 con 0x559adb965f10 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.375+0000 7fa334c00700 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb965f10 0x559adb964c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa334c00700 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb965f10 0x559adb964c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52a16a777c254866 server_cookie=d5ec5bc9c94e5250 in_seq=0 out_seq=0 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa316a00700 10 client.14856 ms_handle_connect on v2:172.21.15.179:6834/1407039746 2024-06-03T01:08:52.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa316a00700 1 -- 172.21.15.94:0/763197943 <== mds.4 v2:172.21.15.179:6834/1407039746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x559adb835530 con 0x559adb965f10 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa316a00700 10 client.14856 handle_command_reply: tid=0 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 client.14856 shutdown 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8db560 msgr2=0x559adb8fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8db560 0x559adb8fd790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa31c000c00 tx=0x7fa31c00be80 comp rx=0 tx=0).stop 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 shutdown_connections 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559adb8e62b0 0x559adb8fdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x559adb965f10 0x559adb964c20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559adb857df0 0x559adb900ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 --2- 172.21.15.94:0/763197943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559adb8db560 0x559adb8fd790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 >> 172.21.15.94:0/763197943 conn(0x559adb808120 msgr2=0x559adb8d5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa316a00700 0 client.14856 ms_handle_reset on v2:172.21.15.179:6834/1407039746 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.376+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 shutdown_connections 2024-06-03T01:08:52.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.377+0000 7fa342067b80 1 -- 172.21.15.94:0/763197943 wait complete. 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.377+0000 7fa342067b80 20 client.14856 trim_cache size 0 max 16384 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 4, 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24478, 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:08:52.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 155.99514786399999, 2024-06-03T01:08:52.385 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:08:52.385 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:08:52.385 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:08:52.385 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 280.03348492100002 2024-06-03T01:08:52.385 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.377+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa3180a2c20 msgr2=0x7fa3180a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.377+0000 7fa337600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa3180a2c20 0x7fa3180a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa328009fd0 tx=0x7fa3280099e0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.377+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa33006ab60 msgr2=0x7fa330101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.377+0000 7fa337600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa33006ab60 0x7fa330101840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa32400e9c0 tx=0x7fa32400ecd0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 shutdown_connections 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa33006ab60 0x7fa330101840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa3180a2c20 0x7fa3180a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa330108520 0x7fa3301022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 --2- 172.21.15.94:0/2946522769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa33006b520 0x7fa330101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 >> 172.21.15.94:0/2946522769 conn(0x7fa330078340 msgr2=0x7fa330100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:52.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 shutdown_connections 2024-06-03T01:08:52.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:52.378+0000 7fa337600700 1 -- 172.21.15.94:0/2946522769 wait complete. 2024-06-03T01:08:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:52 smithi107 ceph-mon[36622]: pgmap v459: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:52.805 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:52 smithi094 ceph-mon[32119]: pgmap v459: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:52 smithi179 ceph-mon[37041]: pgmap v459: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:53.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-03T01:08:53.702 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:54 smithi094 ceph-mon[32119]: pgmap v460: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:54 smithi107 ceph-mon[36622]: pgmap v460: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:54 smithi179 ceph-mon[37041]: pgmap v460: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:56.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 -- 172.21.15.94:0/220785295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e010eac0 msgr2=0x7f73e010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 --2- 172.21.15.94:0/220785295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e010eac0 0x7f73e010eea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009990 tx=0x7f73d0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 -- 172.21.15.94:0/220785295 shutdown_connections 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 --2- 172.21.15.94:0/220785295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73e0109480 0x7f73e0115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 --2- 172.21.15.94:0/220785295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e010eac0 0x7f73e010eea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 --2- 172.21.15.94:0/220785295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73e0108ac0 0x7f73e0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 -- 172.21.15.94:0/220785295 >> 172.21.15.94:0/220785295 conn(0x7f73e0075c60 msgr2=0x7f73e01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 -- 172.21.15.94:0/220785295 shutdown_connections 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.191+0000 7f73e6c00700 1 -- 172.21.15.94:0/220785295 wait complete. 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.192+0000 7f73e6c00700 1 Processor -- start 2024-06-03T01:08:56.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.192+0000 7f73e6c00700 1 -- start start 2024-06-03T01:08:56.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73e6c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73e0108ac0 0x7f73e019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73e6c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e0109480 0x7f73e019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73e6c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73e010eac0 0x7f73e01a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73e6c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0118e10 con 0x7f73e0109480 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73e6c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0118c90 con 0x7f73e010eac0 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73e6c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0118f90 con 0x7f73e0108ac0 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73df400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e0109480 0x7f73e019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73df400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e0109480 0x7f73e019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56302/0 (socket says 172.21.15.94:56302) 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73df400700 1 -- 172.21.15.94:0/925068374 learned_addr learned my addr 172.21.15.94:0/925068374 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73e0108ac0 0x7f73e019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.193+0000 7f73e4c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73e010eac0 0x7f73e01a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73df400700 1 -- 172.21.15.94:0/925068374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73e0108ac0 msgr2=0x7f73e019f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73df400700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73e0108ac0 0x7f73e019f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73df400700 1 -- 172.21.15.94:0/925068374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73e010eac0 msgr2=0x7f73e01a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73df400700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73e010eac0 0x7f73e01a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73df400700 1 -- 172.21.15.94:0/925068374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e01a41e0 con 0x7f73e0109480 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73e4c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73e010eac0 0x7f73e01a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73df400700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e0109480 0x7f73e019f760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00eb30 tx=0x7f73cc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73dcc00700 1 -- 172.21.15.94:0/925068374 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00cbd0 con 0x7f73e0109480 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73dcc00700 1 -- 172.21.15.94:0/925068374 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73cc004500 con 0x7f73e0109480 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e01a44d0 con 0x7f73e0109480 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73dcc00700 1 -- 172.21.15.94:0/925068374 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc010430 con 0x7f73e0109480 2024-06-03T01:08:56.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.194+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01abd50 con 0x7f73e0109480 2024-06-03T01:08:56.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.199+0000 7f73dcc00700 1 -- 172.21.15.94:0/925068374 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f73cc003f00 con 0x7f73e0109480 2024-06-03T01:08:56.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.200+0000 7f73f157db80 1 Processor -- start 2024-06-03T01:08:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.200+0000 7f73f157db80 1 -- start start 2024-06-03T01:08:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73dcc00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73c40a2c20 0x7f73c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73dcc00700 1 -- 172.21.15.94:0/925068374 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f73cc0cf720 con 0x7f73e0109480 2024-06-03T01:08:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73f157db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73f157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73f157db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73f157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b255d69c0 con 0x562b256ce460 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73f157db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b25619ab0 con 0x562b256cdb40 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73f157db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b255d48a0 con 0x562b256c81f0 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73e4c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73c40a2c20 0x7f73c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:46982/0 (socket says 172.21.15.94:46982) 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73dfe00700 1 -- 172.21.15.94:0/3034679404 learned_addr learned my addr 172.21.15.94:0/3034679404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73df400700 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.201+0000 7f73e4c00700 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73dfe00700 1 -- 172.21.15.94:0/3034679404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 msgr2=0x562b256c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73dfe00700 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73dfe00700 1 -- 172.21.15.94:0/3034679404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 msgr2=0x562b256c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73dfe00700 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73dfe00700 1 -- 172.21.15.94:0/3034679404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b255c1f40 con 0x562b256cdb40 2024-06-03T01:08:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73df400700 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:08:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73dfe00700 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256cdf20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009990 tx=0x7f73d0005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23cc41dedec08dd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:56.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73e4c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73c40a2c20 0x7f73c40a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009ec0 tx=0x7f73d8004570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73d5600700 1 -- 172.21.15.94:0/3034679404 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0027070 con 0x562b256cdb40 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73d5600700 1 -- 172.21.15.94:0/3034679404 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d0005550 con 0x562b256cdb40 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.202+0000 7f73d5600700 1 -- 172.21.15.94:0/3034679404 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0003ce0 con 0x562b256cdb40 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 -- 172.21.15.94:0/3034679404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 msgr2=0x562b256cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256cdf20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009990 tx=0x7f73d0005bc0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 -- 172.21.15.94:0/3034679404 shutdown_connections 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256cdf20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 --2- 172.21.15.94:0/3034679404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 -- 172.21.15.94:0/3034679404 >> 172.21.15.94:0/3034679404 conn(0x562b255f4120 msgr2=0x562b256c2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 -- 172.21.15.94:0/3034679404 shutdown_connections 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.203+0000 7f73f157db80 1 -- 172.21.15.94:0/3034679404 wait complete. 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.204+0000 7f73f157db80 1 -- start start 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.204+0000 7f73f157db80 1 Processor -- start 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73f157db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256c81f0 0x562b256d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73f157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256cdb40 0x562b256d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73f157db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73f157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b255b5f10 con 0x562b256cdb40 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73f157db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b25615a70 con 0x562b256c81f0 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73f157db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b256d4de0 con 0x562b256ce460 2024-06-03T01:08:56.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73f157db80 10 client.0 _ll_register_callbacks cb 0x562b255af330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73df400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256cdb40 0x562b256d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256c81f0 0x562b256d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36138/0 (socket says 172.21.15.94:36138) 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 -- 172.21.15.94:0/3803875443 learned_addr learned my addr 172.21.15.94:0/3803875443 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 -- 172.21.15.94:0/3803875443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256c81f0 msgr2=0x562b256d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256c81f0 0x562b256d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 -- 172.21.15.94:0/3803875443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256cdb40 msgr2=0x562b256d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256cdb40 0x562b256d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.205+0000 7f73e4c00700 1 -- 172.21.15.94:0/3803875443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b25568880 con 0x562b256ce460 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73e4c00700 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d1a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73d8003d10 tx=0x7f73d8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e7c4b0b0a560bc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:56.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73ca000700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73ca000700 1 -- 172.21.15.94:0/3803875443 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8028020 con 0x562b256ce460 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d0005550 con 0x562b256ce460 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73ca000700 1 -- 172.21.15.94:0/3803875443 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f73d8024420 con 0x562b256ce460 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73ca000700 1 -- 172.21.15.94:0/3803875443 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73d8033050 con 0x562b256ce460 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73f157db80 10 client.24988 fetch_fsmap learned FSMap version 27 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.206+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b256df520 con 0x562b256ce460 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73ca000700 1 -- 172.21.15.94:0/3803875443 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d0005550 con 0x562b256ce460 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73ca000700 1 -- 172.21.15.94:0/3803875443 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f73d8018760 con 0x562b256ce460 2024-06-03T01:08:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73f157db80 10 client.24988 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:08:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73f157db80 10 client.24988 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T01:08:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73f157db80 20 client.24988 populate_metadata read hostname 'smithi094' 2024-06-03T01:08:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73f157db80 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b256e1b70 0x562b256e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73f157db80 4 client.24988 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:08:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.207+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 --> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b25626fb0 con 0x562b256e1b70 2024-06-03T01:08:56.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.208+0000 7f73dfe00700 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b256e1b70 0x562b256e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.208+0000 7f73dfe00700 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b256e1b70 0x562b256e2fe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3501bbf136038b server_cookie=97c6ece42abbcec8 in_seq=0 out_seq=0 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.208+0000 7f73ca000700 10 client.24988 ms_handle_connect on v2:172.21.15.107:6836/2537813051 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73ca000700 1 -- 172.21.15.94:0/3803875443 <== mds.3 v2:172.21.15.107:6836/2537813051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b25626fb0 con 0x562b256e1b70 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73ca000700 10 client.24988 handle_command_reply: tid=0 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 client.24988 shutdown 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 msgr2=0x562b256d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d1a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73d8003d10 tx=0x7f73d8004b40 comp rx=0 tx=0).stop 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 shutdown_connections 2024-06-03T01:08:56.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256c81f0 0x562b256d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b256e1b70 0x562b256e2fe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d1a00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73ca000700 0 client.24988 ms_handle_reset on v2:172.21.15.107:6836/2537813051 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 --2- 172.21.15.94:0/3803875443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256cdb40 0x562b256d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.209+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 >> 172.21.15.94:0/3803875443 conn(0x562b255f4120 msgr2=0x562b256bc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.210+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 shutdown_connections 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.210+0000 7f73f157db80 1 -- 172.21.15.94:0/3803875443 wait complete. 2024-06-03T01:08:56.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.210+0000 7f73f157db80 20 client.24988 trim_cache size 0 max 16384 2024-06-03T01:08:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 Processor -- start 2024-06-03T01:08:56.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 -- start start 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256c81f0 0x562b256c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b256e1320 con 0x562b256c81f0 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b256e19f0 con 0x562b256cdb40 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73f157db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b256e20c0 con 0x562b256ce460 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256c81f0 0x562b256c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256c81f0 0x562b256c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56328/0 (socket says 172.21.15.94:56328) 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73e4c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73dfe00700 1 -- 172.21.15.94:0/1354044026 learned_addr learned my addr 172.21.15.94:0/1354044026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:56.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.233+0000 7f73df400700 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73dfe00700 1 -- 172.21.15.94:0/1354044026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 msgr2=0x562b256d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73dfe00700 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73dfe00700 1 -- 172.21.15.94:0/1354044026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 msgr2=0x562b256bd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73dfe00700 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256bd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73dfe00700 1 -- 172.21.15.94:0/1354044026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b256c46f0 con 0x562b256c81f0 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73dfe00700 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256c81f0 0x562b256c7cb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73d00098e0 tx=0x7f73d001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20332db37f2e7311 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73d4c00700 1 -- 172.21.15.94:0/1354044026 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0027040 con 0x562b256c81f0 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73d4c00700 1 -- 172.21.15.94:0/1354044026 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d0028040 con 0x562b256c81f0 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.234+0000 7f73d4c00700 1 -- 172.21.15.94:0/1354044026 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0005660 con 0x562b256c81f0 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 -- 172.21.15.94:0/1354044026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256c81f0 msgr2=0x562b256c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256c81f0 0x562b256c7cb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f73d00098e0 tx=0x7f73d001a040 comp rx=0 tx=0).stop 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 -- 172.21.15.94:0/1354044026 shutdown_connections 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256ce460 0x562b256d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256c81f0 0x562b256c7cb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 --2- 172.21.15.94:0/1354044026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 -- 172.21.15.94:0/1354044026 >> 172.21.15.94:0/1354044026 conn(0x562b255f4120 msgr2=0x562b256d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:56.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 -- 172.21.15.94:0/1354044026 shutdown_connections 2024-06-03T01:08:56.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.235+0000 7f73f157db80 1 -- 172.21.15.94:0/1354044026 wait complete. 2024-06-03T01:08:56.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.236+0000 7f73f157db80 1 -- start start 2024-06-03T01:08:56.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.236+0000 7f73f157db80 1 Processor -- start 2024-06-03T01:08:56.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.236+0000 7f73f157db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73f157db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73f157db80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73f157db80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b256e2e60 con 0x562b256ce460 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73f157db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b256e2790 con 0x562b256cdb40 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73f157db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b256e3530 con 0x562b256c81f0 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73f157db80 10 client.0 _ll_register_callbacks cb 0x562b255af330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73df400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256ecaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36168/0 (socket says 172.21.15.94:36168) 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 -- 172.21.15.94:0/2716940443 learned_addr learned my addr 172.21.15.94:0/2716940443 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73e4c00700 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 -- 172.21.15.94:0/2716940443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 msgr2=0x562b256d30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256d30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 -- 172.21.15.94:0/2716940443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 msgr2=0x562b256d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.237+0000 7f73dfe00700 1 -- 172.21.15.94:0/2716940443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b256d3d30 con 0x562b256c81f0 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73dfe00700 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256ecaa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f73d0005a90 tx=0x7f73d0009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=462be80cc070c7b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73ca000700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:08:56.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73ca000700 1 -- 172.21.15.94:0/2716940443 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0027040 con 0x562b256c81f0 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d0028040 con 0x562b256c81f0 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73ca000700 1 -- 172.21.15.94:0/2716940443 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f73d0003a60 con 0x562b256c81f0 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73ca000700 1 -- 172.21.15.94:0/2716940443 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73d0003c60 con 0x562b256c81f0 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73f157db80 10 client.24994 fetch_fsmap learned FSMap version 27 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.238+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562b2574d400 con 0x562b256c81f0 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73ca000700 1 -- 172.21.15.94:0/2716940443 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d0028040 con 0x562b256c81f0 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73ca000700 1 -- 172.21.15.94:0/2716940443 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f73d0021030 con 0x562b256c81f0 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73f157db80 10 client.24994 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:08:56.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73f157db80 10 client.24994 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73f157db80 20 client.24994 populate_metadata read hostname 'smithi094' 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73f157db80 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b257517d0 0x562b257504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73f157db80 4 client.24994 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.239+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 --> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] -- command(tid 0: {"prefix": "status"}) -- 0x562b25621530 con 0x562b257517d0 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.240+0000 7f73df400700 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b257517d0 0x562b257504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.240+0000 7f73df400700 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b257517d0 0x562b257504e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=937afd5e1136b9fe server_cookie=5b51adaf317e6780 in_seq=0 out_seq=0 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.240+0000 7f73ca000700 10 client.24994 ms_handle_connect on v2:172.21.15.107:6836/2537813051 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.240+0000 7f73ca000700 1 -- 172.21.15.94:0/2716940443 <== mds.3 v2:172.21.15.107:6836/2537813051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x562b25621530 con 0x562b257517d0 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.240+0000 7f73ca000700 10 client.24994 handle_command_reply: tid=0 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.240+0000 7f73f157db80 1 client.24994 shutdown 2024-06-03T01:08:56.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 msgr2=0x562b256ecaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256ecaa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f73d0005a90 tx=0x7f73d0009290 comp rx=0 tx=0).stop 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 shutdown_connections 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562b257517d0 0x562b257504e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b256ce460 0x562b256d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b256c81f0 0x562b256ecaa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 --2- 172.21.15.94:0/2716940443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b256cdb40 0x562b256d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73ca000700 0 client.24994 ms_handle_reset on v2:172.21.15.107:6836/2537813051 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 >> 172.21.15.94:0/2716940443 conn(0x562b255f4120 msgr2=0x562b256c2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:56.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 shutdown_connections 2024-06-03T01:08:56.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 1 -- 172.21.15.94:0/2716940443 wait complete. 2024-06-03T01:08:56.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.241+0000 7f73f157db80 20 client.24994 trim_cache size 0 max 16384 2024-06-03T01:08:56.247 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:08:56.247 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:08:56.247 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 3, 2024-06-03T01:08:56.247 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24485, 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 160.872219073, 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 278.15448985400002 2024-06-03T01:08:56.248 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:08:56.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73c40a2c20 msgr2=0x7f73c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73c40a2c20 0x7f73c40a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009ec0 tx=0x7f73d8004570 comp rx=0 tx=0).stop 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e0109480 msgr2=0x7f73e019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e0109480 0x7f73e019f760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00eb30 tx=0x7f73cc00ee40 comp rx=0 tx=0).stop 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 shutdown_connections 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73e010eac0 0x7f73e01a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f73c40a2c20 0x7f73c40a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f73e0108ac0 0x7f73e019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 --2- 172.21.15.94:0/925068374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73e0109480 0x7f73e019f760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:08:56.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 >> 172.21.15.94:0/925068374 conn(0x7f73e0075c60 msgr2=0x7f73e0076180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:08:56.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 shutdown_connections 2024-06-03T01:08:56.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:08:56.242+0000 7f73e6c00700 1 -- 172.21.15.94:0/925068374 wait complete. 2024-06-03T01:08:56.768 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:56 smithi094 ceph-mon[32119]: pgmap v461: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:56.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:56 smithi107 ceph-mon[36622]: pgmap v461: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:56.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:56 smithi179 ceph-mon[37041]: pgmap v461: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:08:57.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-03T01:08:57.635 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:08:57.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:57 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:08:57.796 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:57 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:08:57.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:57 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:08:58.738 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:58 smithi094 ceph-mon[32119]: pgmap v462: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:58.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:58 smithi107 ceph-mon[36622]: pgmap v462: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:58.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:58 smithi179 ceph-mon[37041]: pgmap v462: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:08:59.557 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:59 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:08:59.557 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:59 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:08:59.557 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:08:59 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:08:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:59 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:08:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:59 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:08:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:08:59 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:08:59.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:59 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:08:59.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:59 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:08:59.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:08:59 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:09:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 -- 172.21.15.94:0/2772762857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d6410a670 msgr2=0x7f4d6410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 --2- 172.21.15.94:0/2772762857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d6410a670 0x7f4d6410aa50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4d580099c0 tx=0x7f4d58009cd0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 -- 172.21.15.94:0/2772762857 shutdown_connections 2024-06-03T01:09:00.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 --2- 172.21.15.94:0/2772762857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d64104f80 0x7f4d64111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 --2- 172.21.15.94:0/2772762857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d6410a670 0x7f4d6410aa50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 --2- 172.21.15.94:0/2772762857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d641045c0 0x7f4d64104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 -- 172.21.15.94:0/2772762857 >> 172.21.15.94:0/2772762857 conn(0x7f4d641000d0 msgr2=0x7f4d641024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:00.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 -- 172.21.15.94:0/2772762857 shutdown_connections 2024-06-03T01:09:00.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.110+0000 7f4d6a400700 1 -- 172.21.15.94:0/2772762857 wait complete. 2024-06-03T01:09:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 Processor -- start 2024-06-03T01:09:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 -- start start 2024-06-03T01:09:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d641045c0 0x7f4d641a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d64104f80 0x7f4d641a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d6410a670 0x7f4d6419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d641148c0 con 0x7f4d641045c0 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d64114740 con 0x7f4d64104f80 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.111+0000 7f4d6a400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4d64114a40 con 0x7f4d6410a670 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d641045c0 0x7f4d641a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d641045c0 0x7f4d641a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56364/0 (socket says 172.21.15.94:56364) 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 -- 172.21.15.94:0/1288116560 learned_addr learned my addr 172.21.15.94:0/1288116560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d63e00700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d64104f80 0x7f4d641a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69a00700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d6410a670 0x7f4d6419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 -- 172.21.15.94:0/1288116560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d6410a670 msgr2=0x7f4d6419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d6410a670 0x7f4d6419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 -- 172.21.15.94:0/1288116560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d64104f80 msgr2=0x7f4d641a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d64104f80 0x7f4d641a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.112+0000 7f4d69000700 1 -- 172.21.15.94:0/1288116560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6419b220 con 0x7f4d641045c0 2024-06-03T01:09:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.113+0000 7f4d69000700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d641045c0 0x7f4d641a08b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006010 tx=0x7f4d58003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.113+0000 7f4d61600700 1 -- 172.21.15.94:0/1288116560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58003680 con 0x7f4d641045c0 2024-06-03T01:09:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.113+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6419b4b0 con 0x7f4d641045c0 2024-06-03T01:09:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.113+0000 7f4d61600700 1 -- 172.21.15.94:0/1288116560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d58003e50 con 0x7f4d641045c0 2024-06-03T01:09:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.113+0000 7f4d61600700 1 -- 172.21.15.94:0/1288116560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800fe10 con 0x7f4d641045c0 2024-06-03T01:09:00.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.113+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d641a7780 con 0x7f4d641045c0 2024-06-03T01:09:00.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.116+0000 7f4d61600700 1 -- 172.21.15.94:0/1288116560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d5800f460 con 0x7f4d641045c0 2024-06-03T01:09:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 Processor -- start 2024-06-03T01:09:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d61600700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4d4c0a2dc0 0x7f4d4c0a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 -- start start 2024-06-03T01:09:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d61600700 1 -- 172.21.15.94:0/1288116560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4d580d0090 con 0x7f4d641045c0 2024-06-03T01:09:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597ca0ab9c0 con 0x5597ca1a7520 2024-06-03T01:09:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597ca0eeab0 con 0x5597ca118df0 2024-06-03T01:09:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5597ca0a98a0 con 0x5597ca19e910 2024-06-03T01:09:00.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.118+0000 7f4d69a00700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4d4c0a2dc0 0x7f4d4c0a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47044/0 (socket says 172.21.15.94:47044) 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 -- 172.21.15.94:0/2852045031 learned_addr learned my addr 172.21.15.94:0/2852045031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69a00700 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d63e00700 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 -- 172.21.15.94:0/2852045031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 msgr2=0x5597ca19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 -- 172.21.15.94:0/2852045031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 msgr2=0x5597ca1a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69000700 1 -- 172.21.15.94:0/2852045031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ca096f40 con 0x5597ca118df0 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d63e00700 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.119+0000 7f4d69a00700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4d4c0a2dc0 0x7f4d4c0a5280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00f870 tx=0x7f4d5c00fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.120+0000 7f4d69000700 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca19e3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d58098f70 tx=0x7f4d5809c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91bb452474860b14 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.120+0000 7f4d56000700 1 -- 172.21.15.94:0/2852045031 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5809cca0 con 0x5597ca118df0 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.120+0000 7f4d56000700 1 -- 172.21.15.94:0/2852045031 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5809c5a0 con 0x5597ca118df0 2024-06-03T01:09:00.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.120+0000 7f4d56000700 1 -- 172.21.15.94:0/2852045031 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5809ca60 con 0x5597ca118df0 2024-06-03T01:09:00.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 -- 172.21.15.94:0/2852045031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 msgr2=0x5597ca19e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca19e3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d58098f70 tx=0x7f4d5809c410 comp rx=0 tx=0).stop 2024-06-03T01:09:00.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 -- 172.21.15.94:0/2852045031 shutdown_connections 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca19e3d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/2852045031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 -- 172.21.15.94:0/2852045031 >> 172.21.15.94:0/2852045031 conn(0x5597ca0c9120 msgr2=0x5597ca1987d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 -- 172.21.15.94:0/2852045031 shutdown_connections 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 -- 172.21.15.94:0/2852045031 wait complete. 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.121+0000 7f4d762e4b80 1 -- start start 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 1 Processor -- start 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca1a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597ca08af10 con 0x5597ca1a7520 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597ca0eaa70 con 0x5597ca118df0 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5597ca0eeab0 con 0x5597ca19e910 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d762e4b80 10 client.0 _ll_register_callbacks cb 0x5597ca0886d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.122+0000 7f4d69000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca1a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca1a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47056/0 (socket says 172.21.15.94:47056) 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 -- 172.21.15.94:0/3899267572 learned_addr learned my addr 172.21.15.94:0/3899267572 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d63e00700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69a00700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 -- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 msgr2=0x5597ca1a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 -- 172.21.15.94:0/3899267572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 msgr2=0x5597ca1a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 -- 172.21.15.94:0/3899267572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ca03d880 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d63e00700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d69000700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca1a6d20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800f820 tx=0x7f4d580a0a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320adc3d85d7d8e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d4aa00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3899267572 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d580ba060 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.123+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3899267572 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4d5809c800 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5809c5a0 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3899267572 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d580b4030 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 10 client.25001 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597ca1b44f0 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3899267572 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5809c5a0 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3899267572 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4d580b5350 con 0x5597ca118df0 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 10 client.25001 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 10 client.25001 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T01:09:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 20 client.25001 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca1b69c0 0x5597ca1b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 4 client.25001 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:09:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.124+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 --> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597ca0fbfb0 con 0x5597ca1b69c0 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.125+0000 7f4d63e00700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca1b69c0 0x5597ca1b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.126+0000 7f4d63e00700 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca1b69c0 0x5597ca1b7e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e997b0dba55d33b1 server_cookie=41b3fbe797ef9169 in_seq=0 out_seq=0 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.126+0000 7f4d4aa00700 10 client.25001 ms_handle_connect on v2:172.21.15.179:6836/2233357781 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.126+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3899267572 <== mds.2 v2:172.21.15.179:6836/2233357781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597ca0fbfb0 con 0x5597ca1b69c0 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.126+0000 7f4d4aa00700 10 client.25001 handle_command_reply: tid=0 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.126+0000 7f4d762e4b80 1 client.25001 shutdown 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 msgr2=0x5597ca1a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca1a6d20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800f820 tx=0x7f4d580a0a00 comp rx=0 tx=0).stop 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 shutdown_connections 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca1b69c0 0x5597ca1b7e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca118df0 0x5597ca1a6d20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca1a7520 0x5597ca1a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d4aa00700 0 client.25001 ms_handle_reset on v2:172.21.15.179:6836/2233357781 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3899267572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 >> 172.21.15.94:0/3899267572 conn(0x5597ca0c9120 msgr2=0x5597ca1983c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 shutdown_connections 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3899267572 wait complete. 2024-06-03T01:09:00.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.127+0000 7f4d762e4b80 20 client.25001 trim_cache size 0 max 16384 2024-06-03T01:09:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.150+0000 7f4d762e4b80 1 Processor -- start 2024-06-03T01:09:00.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.150+0000 7f4d762e4b80 1 -- start start 2024-06-03T01:09:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.150+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca118df0 0x5597ca198910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.150+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.150+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca19f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.150+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597ca1b6170 con 0x5597ca118df0 2024-06-03T01:09:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597ca1b6840 con 0x5597ca1a7520 2024-06-03T01:09:00.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5597ca1b6f10 con 0x5597ca19e910 2024-06-03T01:09:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d69000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca118df0 0x5597ca198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d63e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d69a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d69000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca118df0 0x5597ca198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56386/0 (socket says 172.21.15.94:56386) 2024-06-03T01:09:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d63e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36210/0 (socket says 172.21.15.94:36210) 2024-06-03T01:09:00.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d69000700 1 -- 172.21.15.94:0/3241952353 learned_addr learned my addr 172.21.15.94:0/3241952353 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d63e00700 1 -- 172.21.15.94:0/3241952353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 msgr2=0x5597ca19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d63e00700 1 --2- 172.21.15.94:0/3241952353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d63e00700 1 -- 172.21.15.94:0/3241952353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca118df0 msgr2=0x5597ca198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d63e00700 1 --2- 172.21.15.94:0/3241952353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca118df0 0x5597ca198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.151+0000 7f4d63e00700 1 -- 172.21.15.94:0/3241952353 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ca19e460 con 0x5597ca19e910 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.152+0000 7f4d63e00700 1 --2- 172.21.15.94:0/3241952353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5597ca1b85d0 tx=0x7f4d50009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8d7ad204c2ae26 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.152+0000 7f4d55600700 1 -- 172.21.15.94:0/3241952353 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011dc0 con 0x5597ca19e910 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.152+0000 7f4d55600700 1 -- 172.21.15.94:0/3241952353 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d50004530 con 0x5597ca19e910 2024-06-03T01:09:00.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.152+0000 7f4d55600700 1 -- 172.21.15.94:0/3241952353 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50005310 con 0x5597ca19e910 2024-06-03T01:09:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3241952353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 msgr2=0x5597ca1a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3241952353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5597ca1b85d0 tx=0x7f4d50009260 comp rx=0 tx=0).stop 2024-06-03T01:09:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3241952353 shutdown_connections 2024-06-03T01:09:00.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3241952353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3241952353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca118df0 0x5597ca198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3241952353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca19e910 0x5597ca1a4590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3241952353 >> 172.21.15.94:0/3241952353 conn(0x5597ca0c9120 msgr2=0x5597ca196e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3241952353 shutdown_connections 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.153+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3241952353 wait complete. 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 -- start start 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 Processor -- start 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca118df0 0x5597ca1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca19e910 0x5597ca1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca1c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597ca1b7cb0 con 0x5597ca19e910 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597ca1b75e0 con 0x5597ca1a7520 2024-06-03T01:09:00.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5597ca1b8380 con 0x5597ca118df0 2024-06-03T01:09:00.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.154+0000 7f4d762e4b80 10 client.0 _ll_register_callbacks cb 0x5597ca0886d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d63e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca19e910 0x5597ca1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca1c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d63e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca19e910 0x5597ca1a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56388/0 (socket says 172.21.15.94:56388) 2024-06-03T01:09:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d63e00700 1 -- 172.21.15.94:0/3158841742 learned_addr learned my addr 172.21.15.94:0/3158841742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca118df0 0x5597ca1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69a00700 1 -- 172.21.15.94:0/3158841742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca118df0 msgr2=0x5597ca1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69a00700 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca118df0 0x5597ca1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69a00700 1 -- 172.21.15.94:0/3158841742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca19e910 msgr2=0x5597ca1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69a00700 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca19e910 0x5597ca1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69a00700 1 -- 172.21.15.94:0/3158841742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ca1c0890 con 0x5597ca1a7520 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d69a00700 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca1c02a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5597ca0eebd0 tx=0x7f4d5c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=335efaff73e60459 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.155+0000 7f4d4aa00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3158841742 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c008b70 con 0x5597ca1a7520 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3158841742 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f4d5c008d10 con 0x5597ca1a7520 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d50004530 con 0x5597ca1a7520 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3158841742 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c039050 con 0x5597ca1a7520 2024-06-03T01:09:00.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d762e4b80 10 client.25007 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597ca1c1b80 con 0x5597ca1a7520 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3158841742 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d50004530 con 0x5597ca1a7520 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.156+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3158841742 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4d5c01a560 con 0x5597ca1a7520 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.157+0000 7f4d762e4b80 10 client.25007 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.157+0000 7f4d762e4b80 10 client.25007 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.157+0000 7f4d762e4b80 20 client.25007 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.157+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca226950 0x5597ca225660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.157+0000 7f4d762e4b80 4 client.25007 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.157+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 --> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] -- command(tid 0: {"prefix": "status"}) -- 0x5597ca0f6530 con 0x5597ca226950 2024-06-03T01:09:00.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.157+0000 7f4d69000700 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca226950 0x5597ca225660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d69000700 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca226950 0x5597ca225660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c798fce77ba6eea5 server_cookie=74612fe4df9bc6d1 in_seq=0 out_seq=0 2024-06-03T01:09:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d4aa00700 10 client.25007 ms_handle_connect on v2:172.21.15.179:6836/2233357781 2024-06-03T01:09:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d4aa00700 1 -- 172.21.15.94:0/3158841742 <== mds.2 v2:172.21.15.179:6836/2233357781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5597ca0f6530 con 0x5597ca226950 2024-06-03T01:09:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d4aa00700 10 client.25007 handle_command_reply: tid=0 2024-06-03T01:09:00.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 client.25007 shutdown 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 msgr2=0x5597ca1c02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca1c02a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5597ca0eebd0 tx=0x7f4d5c004b40 comp rx=0 tx=0).stop 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 shutdown_connections 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5597ca226950 0x5597ca225660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597ca1a7520 0x5597ca1c02a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597ca118df0 0x5597ca1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 --2- 172.21.15.94:0/3158841742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597ca19e910 0x5597ca1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d4aa00700 0 client.25007 ms_handle_reset on v2:172.21.15.179:6836/2233357781 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 >> 172.21.15.94:0/3158841742 conn(0x5597ca0c9120 msgr2=0x5597ca19f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 shutdown_connections 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 1 -- 172.21.15.94:0/3158841742 wait complete. 2024-06-03T01:09:00.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.158+0000 7f4d762e4b80 20 client.25007 trim_cache size 0 max 16384 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24490, 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 165.820346771, 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:09:00.162 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:09:00.163 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 284.87990945899998 2024-06-03T01:09:00.163 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:09:00.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4d4c0a2dc0 msgr2=0x7f4d4c0a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4d4c0a2dc0 0x7f4d4c0a5280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00f870 tx=0x7f4d5c00fc30 comp rx=0 tx=0).stop 2024-06-03T01:09:00.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d641045c0 msgr2=0x7f4d641a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:00.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d641045c0 0x7f4d641a08b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4d58006010 tx=0x7f4d58003a90 comp rx=0 tx=0).stop 2024-06-03T01:09:00.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 shutdown_connections 2024-06-03T01:09:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f4d4c0a2dc0 0x7f4d4c0a5280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4d6410a670 0x7f4d6419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d641045c0 0x7f4d641a08b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 --2- 172.21.15.94:0/1288116560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d64104f80 0x7f4d641a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 >> 172.21.15.94:0/1288116560 conn(0x7f4d641000d0 msgr2=0x7f4d64101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 shutdown_connections 2024-06-03T01:09:00.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:00.159+0000 7f4d6a400700 1 -- 172.21.15.94:0/1288116560 wait complete. 2024-06-03T01:09:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:00 smithi107 ceph-mon[36622]: pgmap v463: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:00.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:00 smithi094 ceph-mon[32119]: pgmap v463: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:00.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:00 smithi179 ceph-mon[37041]: pgmap v463: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:01.222 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-03T01:09:01.669 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:02 smithi094 ceph-mon[32119]: pgmap v464: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:02 smithi107 ceph-mon[36622]: pgmap v464: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:02.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:02 smithi179 ceph-mon[37041]: pgmap v464: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:03.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.993+0000 7f19fd600700 1 -- 172.21.15.94:0/332946707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8104680 msgr2=0x7f19f8104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:03.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.993+0000 7f19fd600700 1 --2- 172.21.15.94:0/332946707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8104680 0x7f19f8104b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009990 tx=0x7f19e0009ca0 comp rx=0 tx=0).stop 2024-06-03T01:09:03.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.993+0000 7f19fd600700 1 -- 172.21.15.94:0/332946707 shutdown_connections 2024-06-03T01:09:03.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.993+0000 7f19fd600700 1 --2- 172.21.15.94:0/332946707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19f8105040 0x7f19f8111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:03.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.993+0000 7f19fd600700 1 --2- 172.21.15.94:0/332946707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f810a680 0x7f19f810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:03.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.993+0000 7f19fd600700 1 --2- 172.21.15.94:0/332946707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8104680 0x7f19f8104b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:03.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.993+0000 7f19fd600700 1 -- 172.21.15.94:0/332946707 >> 172.21.15.94:0/332946707 conn(0x7f19f8100170 msgr2=0x7f19f8102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.994+0000 7f19fd600700 1 -- 172.21.15.94:0/332946707 shutdown_connections 2024-06-03T01:09:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.994+0000 7f19fd600700 1 -- 172.21.15.94:0/332946707 wait complete. 2024-06-03T01:09:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 Processor -- start 2024-06-03T01:09:03.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 -- start start 2024-06-03T01:09:03.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f8104680 0x7f19f819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:03.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8105040 0x7f19f819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19f810a680 0x7f19f819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81149d0 con 0x7f19f8105040 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8114850 con 0x7f19f810a680 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.995+0000 7f19fd600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8114b50 con 0x7f19f8104680 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8105040 0x7f19f819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f8104680 0x7f19f819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f8104680 0x7f19f819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36240/0 (socket says 172.21.15.94:36240) 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 -- 172.21.15.94:0/2706811023 learned_addr learned my addr 172.21.15.94:0/2706811023 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f7400700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19f810a680 0x7f19f819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 -- 172.21.15.94:0/2706811023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19f810a680 msgr2=0x7f19f819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19f810a680 0x7f19f819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 -- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8105040 msgr2=0x7f19f819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8105040 0x7f19f819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 -- 172.21.15.94:0/2706811023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f819fe30 con 0x7f19f8104680 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.996+0000 7f19f6a00700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f8104680 0x7f19f819ae20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e9c0 tx=0x7f19e800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.997+0000 7f19ef400700 1 -- 172.21.15.94:0/2706811023 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800cbd0 con 0x7f19f8104680 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.997+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f81a0120 con 0x7f19f8104680 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.997+0000 7f19ef400700 1 -- 172.21.15.94:0/2706811023 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e800cd70 con 0x7f19f8104680 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.997+0000 7f19ef400700 1 -- 172.21.15.94:0/2706811023 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8010430 con 0x7f19f8104680 2024-06-03T01:09:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:03.997+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f81a79a0 con 0x7f19f8104680 2024-06-03T01:09:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.000+0000 7f19ef400700 1 -- 172.21.15.94:0/2706811023 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19e8004750 con 0x7f19f8104680 2024-06-03T01:09:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.001+0000 7f1a07f0cb80 1 Processor -- start 2024-06-03T01:09:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.001+0000 7f19ef400700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f19dc0a2c20 0x7f19dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.001+0000 7f1a07f0cb80 1 -- start start 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f19ef400700 1 -- 172.21.15.94:0/2706811023 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f19e80cf630 con 0x7f19f8104680 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f19f7400700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f19dc0a2c20 0x7f19dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edc0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edc0560 0x559c3edc09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edcb2b0 0x559c3edcd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559c3eccf9c0 con 0x559c3edcb2b0 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c3ed12ab0 con 0x559c3edc0560 2024-06-03T01:09:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c3eccd8a0 con 0x559c3ed3cdf0 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f19efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edc0560 0x559c3edc09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f19efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edc0560 0x559c3edc09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47104/0 (socket says 172.21.15.94:47104) 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f19efe00700 1 -- 172.21.15.94:0/627660867 learned_addr learned my addr 172.21.15.94:0/627660867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.002+0000 7f19f6a00700 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edc0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19efe00700 1 -- 172.21.15.94:0/627660867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 msgr2=0x559c3edc0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19efe00700 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edc0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19efe00700 1 -- 172.21.15.94:0/627660867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edcb2b0 msgr2=0x559c3edcd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19efe00700 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edcb2b0 0x559c3edcd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19efe00700 1 -- 172.21.15.94:0/627660867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3ecbaf40 con 0x559c3edc0560 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19efe00700 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edc0560 0x559c3edc09e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009990 tx=0x7f19e0005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7fc112eababd518 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:04.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19f7400700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f19dc0a2c20 0x7f19dc0a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19f00093f0 tx=0x7f19f0009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19e7e00700 1 -- 172.21.15.94:0/627660867 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000bce0 con 0x559c3edc0560 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19e7e00700 1 -- 172.21.15.94:0/627660867 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e000be80 con 0x559c3edc0560 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.003+0000 7f19e7e00700 1 -- 172.21.15.94:0/627660867 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000f460 con 0x559c3edc0560 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/627660867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edc0560 msgr2=0x559c3edc09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edc0560 0x559c3edc09e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009990 tx=0x7f19e0005bc0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/627660867 shutdown_connections 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edcb2b0 0x559c3edcd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edc0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/627660867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edc0560 0x559c3edc09e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/627660867 >> 172.21.15.94:0/627660867 conn(0x559c3eced120 msgr2=0x559c3edbc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.004+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/627660867 shutdown_connections 2024-06-03T01:09:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.005+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/627660867 wait complete. 2024-06-03T01:09:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.005+0000 7f1a07f0cb80 1 -- start start 2024-06-03T01:09:04.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 1 Processor -- start 2024-06-03T01:09:04.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edc7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edcb2b0 0x559c3edc8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559c3ecaef10 con 0x559c3edc0560 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c3ed0ea70 con 0x559c3edcb2b0 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c3edcdf10 con 0x559c3ed3cdf0 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f1a07f0cb80 10 client.0 _ll_register_callbacks cb 0x559c3ecac6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f19f7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edcb2b0 0x559c3edc8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f19efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edc7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f19f7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edcb2b0 0x559c3edc8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47116/0 (socket says 172.21.15.94:47116) 2024-06-03T01:09:04.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f19f7400700 1 -- 172.21.15.94:0/1896536390 learned_addr learned my addr 172.21.15.94:0/1896536390 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.006+0000 7f19f6a00700 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.007+0000 7f19f7400700 1 -- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 msgr2=0x559c3edca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.007+0000 7f19f7400700 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.007+0000 7f19f7400700 1 -- 172.21.15.94:0/1896536390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 msgr2=0x559c3edc7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.007+0000 7f19f7400700 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edc7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.007+0000 7f19f7400700 1 -- 172.21.15.94:0/1896536390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3ec61880 con 0x559c3edcb2b0 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.007+0000 7f19f7400700 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edcb2b0 0x559c3edc8220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19f0017e30 tx=0x7f19f0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4462530294e792f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.007+0000 7f19e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.008+0000 7f19e4c00700 1 -- 172.21.15.94:0/1896536390 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0028060 con 0x559c3edcb2b0 2024-06-03T01:09:04.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.008+0000 7f19e4c00700 1 -- 172.21.15.94:0/1896536390 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f19f0019b10 con 0x559c3edcb2b0 2024-06-03T01:09:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.008+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c3ed1ffb0 con 0x559c3edcb2b0 2024-06-03T01:09:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.008+0000 7f19e4c00700 1 -- 172.21.15.94:0/1896536390 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19f002f050 con 0x559c3edcb2b0 2024-06-03T01:09:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.008+0000 7f1a07f0cb80 10 client.25019 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:04.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.008+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c3edd8650 con 0x559c3edcb2b0 2024-06-03T01:09:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f19e4c00700 1 -- 172.21.15.94:0/1896536390 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559c3ed1ffb0 con 0x559c3edcb2b0 2024-06-03T01:09:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f19e4c00700 1 -- 172.21.15.94:0/1896536390 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19f00163c0 con 0x559c3edcb2b0 2024-06-03T01:09:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f1a07f0cb80 10 client.25019 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f1a07f0cb80 10 client.25019 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T01:09:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f1a07f0cb80 20 client.25019 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3eddac90 0x559c3eddc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f1a07f0cb80 4 client.25019 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.009+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 --> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c3ed1a530 con 0x559c3eddac90 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.010+0000 7f19f6a00700 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3eddac90 0x559c3eddc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.010+0000 7f19f6a00700 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3eddac90 0x559c3eddc100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91a00335f9e7f087 server_cookie=d89d5f09a5230593 in_seq=0 out_seq=0 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.010+0000 7f19e4c00700 10 client.25019 ms_handle_connect on v2:172.21.15.179:6838/544081992 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.011+0000 7f19e4c00700 1 -- 172.21.15.94:0/1896536390 <== mds.1 v2:172.21.15.179:6838/544081992 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c3ed1a530 con 0x559c3eddac90 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.011+0000 7f19e4c00700 10 client.25019 handle_command_reply: tid=0 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.011+0000 7f1a07f0cb80 1 client.25019 shutdown 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.011+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edcb2b0 msgr2=0x559c3edc8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.011+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edcb2b0 0x559c3edc8220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19f0017e30 tx=0x7f19f0004ad0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.011+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 shutdown_connections 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.011+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3eddac90 0x559c3eddc100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3edcb2b0 0x559c3edc8220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3ed3cdf0 0x559c3edca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1896536390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edc7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 >> 172.21.15.94:0/1896536390 conn(0x559c3eced120 msgr2=0x559c3edbb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f19e4c00700 0 client.25019 ms_handle_reset on v2:172.21.15.179:6838/544081992 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 shutdown_connections 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1896536390 wait complete. 2024-06-03T01:09:04.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.012+0000 7f1a07f0cb80 20 client.25019 trim_cache size 0 max 16384 2024-06-03T01:09:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 Processor -- start 2024-06-03T01:09:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 -- start start 2024-06-03T01:09:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edda5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edc7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559c3edda440 con 0x559c3edc0560 2024-06-03T01:09:04.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c3eddab10 con 0x559c3ed3cdf0 2024-06-03T01:09:04.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.035+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c3eddb1e0 con 0x559c3edcb2b0 2024-06-03T01:09:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56416/0 (socket says 172.21.15.94:56416) 2024-06-03T01:09:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 -- 172.21.15.94:0/1093809091 learned_addr learned my addr 172.21.15.94:0/1093809091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19f7400700 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edc7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19f6a00700 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edda5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 -- 172.21.15.94:0/1093809091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 msgr2=0x559c3edc7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edc7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 -- 172.21.15.94:0/1093809091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 msgr2=0x559c3edda5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edda5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 -- 172.21.15.94:0/1093809091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3edc6250 con 0x559c3edc0560 2024-06-03T01:09:04.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.036+0000 7f19efe00700 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcc6a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009ce0 tx=0x7f19e00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6b23e47e57a496 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:04.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.037+0000 7f19e7400700 1 -- 172.21.15.94:0/1093809091 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0033070 con 0x559c3edc0560 2024-06-03T01:09:04.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.037+0000 7f19e7400700 1 -- 172.21.15.94:0/1093809091 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e000b610 con 0x559c3edc0560 2024-06-03T01:09:04.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.037+0000 7f19e7400700 1 -- 172.21.15.94:0/1093809091 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e002e480 con 0x559c3edc0560 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.037+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1093809091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 msgr2=0x559c3edcc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.037+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcc6a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009ce0 tx=0x7f19e00039f0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.037+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1093809091 shutdown_connections 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.037+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edda5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.038+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edc7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.038+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/1093809091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcc6a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.038+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1093809091 >> 172.21.15.94:0/1093809091 conn(0x559c3eced120 msgr2=0x559c3edcd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.038+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1093809091 shutdown_connections 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.038+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/1093809091 wait complete. 2024-06-03T01:09:04.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.038+0000 7f1a07f0cb80 1 -- start start 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.038+0000 7f1a07f0cb80 1 Processor -- start 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edcd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f1a07f0cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edcc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559c3eddbf80 con 0x559c3edc0560 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559c3eddb8b0 con 0x559c3ed3cdf0 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f1a07f0cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c3eddc650 con 0x559c3edcb2b0 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f1a07f0cb80 10 client.0 _ll_register_callbacks cb 0x559c3ecac6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f19efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f19efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56430/0 (socket says 172.21.15.94:56430) 2024-06-03T01:09:04.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f19efe00700 1 -- 172.21.15.94:0/769004134 learned_addr learned my addr 172.21.15.94:0/769004134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.039+0000 7f19f7400700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edcc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19f6a00700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edcd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19efe00700 1 -- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 msgr2=0x559c3edcc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19efe00700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edcc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19efe00700 1 -- 172.21.15.94:0/769004134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 msgr2=0x559c3edcd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19efe00700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edcd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19efe00700 1 -- 172.21.15.94:0/769004134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3ede4570 con 0x559c3edc0560 2024-06-03T01:09:04.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19f7400700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edcc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19efe00700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcba90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19e00059a0 tx=0x7f19e0005110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded8f7f6719e45d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19e4c00700 1 -- 172.21.15.94:0/769004134 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e003b060 con 0x559c3edc0560 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f19e4c00700 1 -- 172.21.15.94:0/769004134 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f19e00055f0 con 0x559c3edc0560 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.040+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e000b610 con 0x559c3edc0560 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f19e4c00700 1 -- 172.21.15.94:0/769004134 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f19e0033070 con 0x559c3edc0560 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 10 client.14904 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c3ede5850 con 0x559c3edc0560 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f19e4c00700 1 -- 172.21.15.94:0/769004134 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e000b610 con 0x559c3edc0560 2024-06-03T01:09:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f19e4c00700 1 -- 172.21.15.94:0/769004134 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f19e003e080 con 0x559c3edc0560 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 10 client.14904 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 10 client.14904 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 20 client.14904 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3ee52540 0x559c3ee51250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 4 client.14904 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.041+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 --> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] -- command(tid 0: {"prefix": "status"}) -- 0x559c3eb94200 con 0x559c3ee52540 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f19f6a00700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3ee52540 0x559c3ee51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f19f6a00700 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3ee52540 0x559c3ee51250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17a1b47a1418f328 server_cookie=50a984818fcfeb45 in_seq=0 out_seq=0 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f19e4c00700 10 client.14904 ms_handle_connect on v2:172.21.15.179:6838/544081992 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f19e4c00700 1 -- 172.21.15.94:0/769004134 <== mds.1 v2:172.21.15.179:6838/544081992 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x559c3eb94200 con 0x559c3ee52540 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f19e4c00700 10 client.14904 handle_command_reply: tid=0 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f1a07f0cb80 1 client.14904 shutdown 2024-06-03T01:09:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 msgr2=0x559c3edcba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.042+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcba90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19e00059a0 tx=0x7f19e0005110 comp rx=0 tx=0).stop 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 shutdown_connections 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x559c3ee52540 0x559c3ee51250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559c3ed3cdf0 0x559c3edcd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3edcb2b0 0x559c3edcc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f19e4c00700 0 client.14904 ms_handle_reset on v2:172.21.15.179:6838/544081992 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 --2- 172.21.15.94:0/769004134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559c3edc0560 0x559c3edcba90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 >> 172.21.15.94:0/769004134 conn(0x559c3eced120 msgr2=0x559c3edc7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 shutdown_connections 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 1 -- 172.21.15.94:0/769004134 wait complete. 2024-06-03T01:09:04.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.043+0000 7f1a07f0cb80 20 client.14904 trim_cache size 0 max 16384 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24505, 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 170.71632155200001, 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 283.02330031299999 2024-06-03T01:09:04.048 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f19dc0a2c20 msgr2=0x7f19dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f19dc0a2c20 0x7f19dc0a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19f00093f0 tx=0x7f19f0009920 comp rx=0 tx=0).stop 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f8104680 msgr2=0x7f19f819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f8104680 0x7f19f819ae20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e9c0 tx=0x7f19e800ecd0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 shutdown_connections 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19f810a680 0x7f19f819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f19dc0a2c20 0x7f19dc0a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19f8104680 0x7f19f819ae20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 --2- 172.21.15.94:0/2706811023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f19f8105040 0x7f19f819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 >> 172.21.15.94:0/2706811023 conn(0x7f19f8100170 msgr2=0x7f19f81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 shutdown_connections 2024-06-03T01:09:04.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:04.044+0000 7f19fd600700 1 -- 172.21.15.94:0/2706811023 wait complete. 2024-06-03T01:09:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:04 smithi107 ceph-mon[36622]: pgmap v465: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:04 smithi094 ceph-mon[32119]: pgmap v465: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:04 smithi179 ceph-mon[37041]: pgmap v465: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:05.138 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T01:09:05.593 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:06.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:06 smithi094 ceph-mon[32119]: pgmap v466: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:06 smithi107 ceph-mon[36622]: pgmap v466: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:06 smithi179 ceph-mon[37041]: pgmap v466: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 -- 172.21.15.94:0/2685323851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 msgr2=0x7f7cf0115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2685323851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf0115990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800b0a0 tx=0x7f7ce800b3b0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 -- 172.21.15.94:0/2685323851 shutdown_connections 2024-06-03T01:09:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2685323851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf0115990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2685323851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf010eac0 0x7f7cf010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2685323851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cf0108ac0 0x7f7cf0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 -- 172.21.15.94:0/2685323851 >> 172.21.15.94:0/2685323851 conn(0x7f7cf0075c60 msgr2=0x7f7cf01072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 -- 172.21.15.94:0/2685323851 shutdown_connections 2024-06-03T01:09:08.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.005+0000 7f7cf5600700 1 -- 172.21.15.94:0/2685323851 wait complete. 2024-06-03T01:09:08.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 Processor -- start 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 -- start start 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cf0108ac0 0x7f7cf019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf010eac0 0x7f7cf01a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0118ea0 con 0x7f7cf0109480 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0118d20 con 0x7f7cf0108ac0 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.007+0000 7f7cf5600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf0119020 con 0x7f7cf010eac0 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7cee000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7cee000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33462/0 (socket says 172.21.15.94:33462) 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7cee000700 1 -- 172.21.15.94:0/2933562380 learned_addr learned my addr 172.21.15.94:0/2933562380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7ceea00700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cf0108ac0 0x7f7cf019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7cef400700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf010eac0 0x7f7cf01a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7ceea00700 1 -- 172.21.15.94:0/2933562380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf010eac0 msgr2=0x7f7cf01a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7ceea00700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf010eac0 0x7f7cf01a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7ceea00700 1 -- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 msgr2=0x7f7cf019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7ceea00700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7ceea00700 1 -- 172.21.15.94:0/2933562380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf01a4220 con 0x7f7cf0108ac0 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7cef400700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf010eac0 0x7f7cf01a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.008+0000 7f7cee000700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:08.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.009+0000 7f7ceea00700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cf0108ac0 0x7f7cf019f210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000e9f0 tx=0x7f7ce000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.009+0000 7f7ce7400700 1 -- 172.21.15.94:0/2933562380 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000c9c0 con 0x7f7cf0108ac0 2024-06-03T01:09:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.009+0000 7f7ce7400700 1 -- 172.21.15.94:0/2933562380 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce000cb60 con 0x7f7cf0108ac0 2024-06-03T01:09:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.009+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf01a4510 con 0x7f7cf0108ac0 2024-06-03T01:09:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.009+0000 7f7ce7400700 1 -- 172.21.15.94:0/2933562380 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0018680 con 0x7f7cf0108ac0 2024-06-03T01:09:08.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.009+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf01abd90 con 0x7f7cf0108ac0 2024-06-03T01:09:08.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.013+0000 7f7ce7400700 1 -- 172.21.15.94:0/2933562380 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ce0010430 con 0x7f7cf0108ac0 2024-06-03T01:09:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 Processor -- start 2024-06-03T01:09:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 -- start start 2024-06-03T01:09:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7ce7400700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7cd40a2c20 0x7f7cd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7ce7400700 1 -- 172.21.15.94:0/2933562380 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7ce00cf4b0 con 0x7f7cf0108ac0 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdf1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7cef400700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7cd40a2c20 0x7f7cd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdf2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa0bd009c0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa0bd43ab0 con 0x55fa0bdf7b40 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.014+0000 7f7d00001b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa0bcfe8a0 con 0x55fa0bdf8460 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7cef400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdf2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7cee000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdf1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7ceea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7cee000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdf1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51284/0 (socket says 172.21.15.94:51284) 2024-06-03T01:09:08.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7ceea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33286/0 (socket says 172.21.15.94:33286) 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7cee000700 1 -- 172.21.15.94:0/3722428815 learned_addr learned my addr 172.21.15.94:0/3722428815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7ceea00700 1 -- 172.21.15.94:0/3722428815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 msgr2=0x55fa0bdf1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7ceea00700 1 --2- 172.21.15.94:0/3722428815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdf1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7ceea00700 1 -- 172.21.15.94:0/3722428815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 msgr2=0x55fa0bdf2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7ceea00700 1 --2- 172.21.15.94:0/3722428815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdf2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.015+0000 7f7ceea00700 1 -- 172.21.15.94:0/3722428815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0bcebf40 con 0x55fa0bdf7b40 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.016+0000 7f7ceea00700 1 --2- 172.21.15.94:0/3722428815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf7f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ce009b630 tx=0x7f7ce0097d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907b824bf762edb7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.016+0000 7f7cef400700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7cd40a2c20 0x7f7cd40a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009850 tx=0x7f7ce8009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.016+0000 7f7cdbe00700 1 -- 172.21.15.94:0/3722428815 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce009b930 con 0x55fa0bdf7b40 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.016+0000 7f7cdbe00700 1 -- 172.21.15.94:0/3722428815 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce009fce0 con 0x55fa0bdf7b40 2024-06-03T01:09:08.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.016+0000 7f7cdbe00700 1 -- 172.21.15.94:0/3722428815 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00860e0 con 0x55fa0bdf7b40 2024-06-03T01:09:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 -- 172.21.15.94:0/3722428815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 msgr2=0x55fa0bdf7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3722428815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf7f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ce009b630 tx=0x7f7ce0097d00 comp rx=0 tx=0).stop 2024-06-03T01:09:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 -- 172.21.15.94:0/3722428815 shutdown_connections 2024-06-03T01:09:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3722428815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf7f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3722428815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdf2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3722428815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdf1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 -- 172.21.15.94:0/3722428815 >> 172.21.15.94:0/3722428815 conn(0x55fa0bd1e120 msgr2=0x55fa0bdecfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 -- 172.21.15.94:0/3722428815 shutdown_connections 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.017+0000 7f7d00001b80 1 -- 172.21.15.94:0/3722428815 wait complete. 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 -- start start 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 Processor -- start 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdfe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdfb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa0bcdff10 con 0x55fa0bdf21f0 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa0bd3fa70 con 0x55fa0bdf7b40 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa0bdfed80 con 0x55fa0bdf8460 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.018+0000 7f7d00001b80 10 client.0 _ll_register_callbacks cb 0x55fa0bcd9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:08.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdfe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdfe200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33488/0 (socket says 172.21.15.94:33488) 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 -- 172.21.15.94:0/1706158994 learned_addr learned my addr 172.21.15.94:0/1706158994 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7cee000700 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdfb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7cef400700 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 -- 172.21.15.94:0/1706158994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 msgr2=0x55fa0bdfba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 -- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 msgr2=0x55fa0bdfb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdfb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 -- 172.21.15.94:0/1706158994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0bc92880 con 0x55fa0bdf21f0 2024-06-03T01:09:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.019+0000 7f7ceea00700 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdfe200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009230 tx=0x7f7ce0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a531a28a26af4b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7cd8c00700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:09:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7cd8c00700 1 -- 172.21.15.94:0/1706158994 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00b3070 con 0x55fa0bdf21f0 2024-06-03T01:09:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7cd8c00700 1 -- 172.21.15.94:0/1706158994 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7ce0010720 con 0x55fa0bdf21f0 2024-06-03T01:09:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce009fce0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7cd8c00700 1 -- 172.21.15.94:0/1706158994 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ce009b930 con 0x55fa0bdf21f0 2024-06-03T01:09:08.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7d00001b80 10 client.14916 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa0be09530 con 0x55fa0bdf21f0 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7cd8c00700 1 -- 172.21.15.94:0/1706158994 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce009fce0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.020+0000 7f7cd8c00700 1 -- 172.21.15.94:0/1706158994 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ce00b4b30 con 0x55fa0bdf21f0 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.021+0000 7f7d00001b80 10 client.14916 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.021+0000 7f7d00001b80 10 client.14916 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.021+0000 7f7d00001b80 20 client.14916 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.021+0000 7f7d00001b80 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be0b690 0x55fa0be0cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.021+0000 7f7d00001b80 4 client.14916 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.021+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa0bd50fb0 con 0x55fa0be0b690 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.021+0000 7f7cee000700 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be0b690 0x55fa0be0cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.022+0000 7f7cee000700 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be0b690 0x55fa0be0cb00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49d8ba43401ff73 server_cookie=631310ad650d06e0 in_seq=0 out_seq=0 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.022+0000 7f7cd8c00700 10 client.14916 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:09:08.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7cd8c00700 1 -- 172.21.15.94:0/1706158994 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa0bd50fb0 con 0x55fa0be0b690 2024-06-03T01:09:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7cd8c00700 10 client.14916 handle_command_reply: tid=0 2024-06-03T01:09:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 client.14916 shutdown 2024-06-03T01:09:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 msgr2=0x55fa0bdfe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdfe200 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009230 tx=0x7f7ce0009ef0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 shutdown_connections 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bdfe200 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be0b690 0x55fa0be0cb00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 --2- 172.21.15.94:0/1706158994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdfb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 >> 172.21.15.94:0/1706158994 conn(0x55fa0bd1e120 msgr2=0x55fa0bde6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7cd8c00700 0 client.14916 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.023+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 shutdown_connections 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.024+0000 7f7d00001b80 1 -- 172.21.15.94:0/1706158994 wait complete. 2024-06-03T01:09:08.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.024+0000 7f7d00001b80 20 client.14916 trim_cache size 0 max 16384 2024-06-03T01:09:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 Processor -- start 2024-06-03T01:09:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 -- start start 2024-06-03T01:09:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bde70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa0be0ae40 con 0x55fa0bdf21f0 2024-06-03T01:09:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa0be0b510 con 0x55fa0bdf8460 2024-06-03T01:09:08.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.047+0000 7f7d00001b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa0be0bbe0 con 0x55fa0bdf7b40 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7ceea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bde70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cee000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33304/0 (socket says 172.21.15.94:33304) 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 -- 172.21.15.94:0/189164343 learned_addr learned my addr 172.21.15.94:0/189164343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 -- 172.21.15.94:0/189164343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf7b40 msgr2=0x55fa0bdf90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 --2- 172.21.15.94:0/189164343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 -- 172.21.15.94:0/189164343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 msgr2=0x55fa0bde70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 --2- 172.21.15.94:0/189164343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bde70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.048+0000 7f7cef400700 1 -- 172.21.15.94:0/189164343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0be0da60 con 0x55fa0bdf8460 2024-06-03T01:09:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.049+0000 7f7cef400700 1 --2- 172.21.15.94:0/189164343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfdd40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800b0a0 tx=0x7f7ce8029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c193cdac9706940 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.049+0000 7f7cdb400700 1 -- 172.21.15.94:0/189164343 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8007c40 con 0x55fa0bdf8460 2024-06-03T01:09:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.049+0000 7f7cdb400700 1 -- 172.21.15.94:0/189164343 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce8007de0 con 0x55fa0bdf8460 2024-06-03T01:09:08.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.049+0000 7f7cdb400700 1 -- 172.21.15.94:0/189164343 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce802b8e0 con 0x55fa0bdf8460 2024-06-03T01:09:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 -- 172.21.15.94:0/189164343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf8460 msgr2=0x55fa0bdfdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 --2- 172.21.15.94:0/189164343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfdd40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800b0a0 tx=0x7f7ce8029000 comp rx=0 tx=0).stop 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 -- 172.21.15.94:0/189164343 shutdown_connections 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 --2- 172.21.15.94:0/189164343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf8460 0x55fa0bdfdd40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 --2- 172.21.15.94:0/189164343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf21f0 0x55fa0bde70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 --2- 172.21.15.94:0/189164343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf7b40 0x55fa0bdf90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 -- 172.21.15.94:0/189164343 >> 172.21.15.94:0/189164343 conn(0x55fa0bd1e120 msgr2=0x55fa0bdf65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 -- 172.21.15.94:0/189164343 shutdown_connections 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 -- 172.21.15.94:0/189164343 wait complete. 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.050+0000 7f7d00001b80 1 -- start start 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 1 Processor -- start 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf21f0 0x55fa0be16240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf7b40 0x55fa0be16780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0be116e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa0be0c980 con 0x55fa0bdf7b40 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa0be0c2b0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa0be0d050 con 0x55fa0bdf8460 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7d00001b80 10 client.0 _ll_register_callbacks cb 0x55fa0bcd9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7cee000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf7b40 0x55fa0be16780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7ceea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf21f0 0x55fa0be16240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7cee000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf7b40 0x55fa0be16780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33506/0 (socket says 172.21.15.94:33506) 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7cee000700 1 -- 172.21.15.94:0/3306864721 learned_addr learned my addr 172.21.15.94:0/3306864721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:08.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.051+0000 7f7cef400700 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0be116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7ceea00700 1 -- 172.21.15.94:0/3306864721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 msgr2=0x55fa0be116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7ceea00700 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0be116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7ceea00700 1 -- 172.21.15.94:0/3306864721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf7b40 msgr2=0x55fa0be16780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7ceea00700 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf7b40 0x55fa0be16780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7ceea00700 1 -- 172.21.15.94:0/3306864721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0be11ef0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7ceea00700 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf21f0 0x55fa0be16240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ce009b630 tx=0x7f7ce009b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a08d7890cef0dd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7cd8c00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7cd8c00700 1 -- 172.21.15.94:0/3306864721 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00b3050 con 0x55fa0bdf21f0 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce8007de0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.052+0000 7f7cd8c00700 1 -- 172.21.15.94:0/3306864721 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7ce00860b0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7cd8c00700 1 -- 172.21.15.94:0/3306864721 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ce0097bd0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 10 client.25043 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa0be76b00 con 0x55fa0bdf21f0 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7cd8c00700 1 -- 172.21.15.94:0/3306864721 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce8007de0 con 0x55fa0bdf21f0 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7cd8c00700 1 -- 172.21.15.94:0/3306864721 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ce0010840 con 0x55fa0bdf21f0 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 10 client.25043 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 10 client.25043 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 20 client.25043 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be7b430 0x55fa0be7a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 4 client.25043 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.053+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "status"}) -- 0x55fa0bd4b530 con 0x55fa0be7b430 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.054+0000 7f7cee000700 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be7b430 0x55fa0be7a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.054+0000 7f7cee000700 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be7b430 0x55fa0be7a140 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aecc59d4990c756a server_cookie=52b006507a1e602f in_seq=0 out_seq=0 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.054+0000 7f7cd8c00700 10 client.25043 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:09:08.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7cd8c00700 1 -- 172.21.15.94:0/3306864721 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55fa0bd4b530 con 0x55fa0be7b430 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7cd8c00700 10 client.25043 handle_command_reply: tid=0 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 client.25043 shutdown 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf21f0 msgr2=0x55fa0be16240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf21f0 0x55fa0be16240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ce009b630 tx=0x7f7ce009b700 comp rx=0 tx=0).stop 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 shutdown_connections 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa0be7b430 0x55fa0be7a140 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa0bdf21f0 0x55fa0be16240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa0bdf8460 0x55fa0be116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 --2- 172.21.15.94:0/3306864721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa0bdf7b40 0x55fa0be16780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 >> 172.21.15.94:0/3306864721 conn(0x55fa0bd1e120 msgr2=0x55fa0bdf8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7cd8c00700 0 client.25043 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 shutdown_connections 2024-06-03T01:09:08.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 1 -- 172.21.15.94:0/3306864721 wait complete. 2024-06-03T01:09:08.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.055+0000 7f7d00001b80 20 client.25043 trim_cache size 0 max 16384 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24524, 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:09:08.062 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 250.37157398799999, 2024-06-03T01:09:08.063 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:09:08.063 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:09:08.063 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:09:08.063 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 274.45771195999998 2024-06-03T01:09:08.063 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7cd40a2c20 msgr2=0x7f7cd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7cd40a2c20 0x7f7cd40a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009850 tx=0x7f7ce8009ca0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cf0108ac0 msgr2=0x7f7cf019f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cf0108ac0 0x7f7cf019f210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000e9f0 tx=0x7f7ce000ed00 comp rx=0 tx=0).stop 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 shutdown_connections 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cf0108ac0 0x7f7cf019f210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7cd40a2c20 0x7f7cd40a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cf010eac0 0x7f7cf01a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 --2- 172.21.15.94:0/2933562380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cf0109480 0x7f7cf019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 >> 172.21.15.94:0/2933562380 conn(0x7f7cf0075c60 msgr2=0x7f7cf0076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 shutdown_connections 2024-06-03T01:09:08.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:08.056+0000 7f7cf5600700 1 -- 172.21.15.94:0/2933562380 wait complete. 2024-06-03T01:09:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:08 smithi107 ceph-mon[36622]: pgmap v467: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:08.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:08 smithi094 ceph-mon[32119]: pgmap v467: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:08 smithi179 ceph-mon[37041]: pgmap v467: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:08.905 INFO:tasks.cephadm:Setup complete, yielding 2024-06-03T01:09:08.905 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:09:08.915 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:09:08.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-03T01:09:09.341 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:10 smithi107 ceph-mon[36622]: pgmap v468: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:10.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:10 smithi094 ceph-mon[32119]: pgmap v468: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:10 smithi179 ceph-mon[37041]: pgmap v468: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 -- 172.21.15.94:0/2295179663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 msgr2=0x7f41b0109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 --2- 172.21.15.94:0/2295179663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 0x7f41b0109870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f41a80099c0 tx=0x7f41a8009cd0 comp rx=0 tx=0).stop 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 -- 172.21.15.94:0/2295179663 shutdown_connections 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 --2- 172.21.15.94:0/2295179663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 0x7f41b0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 --2- 172.21.15.94:0/2295179663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 0x7f41b0109870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 --2- 172.21.15.94:0/2295179663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b0102620 0x7f41b0102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 -- 172.21.15.94:0/2295179663 >> 172.21.15.94:0/2295179663 conn(0x7f41b00fc9e0 msgr2=0x7f41b00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 -- 172.21.15.94:0/2295179663 shutdown_connections 2024-06-03T01:09:11.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.826+0000 7f41b8600700 1 -- 172.21.15.94:0/2295179663 wait complete. 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.827+0000 7f41b8600700 1 Processor -- start 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.827+0000 7f41b8600700 1 -- start start 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b8600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 0x7f41b019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b8600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b0102620 0x7f41b019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b8600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 0x7f41b01a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b8600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41b010d040 con 0x7f41b0102fc0 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b8600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41b010cec0 con 0x7f41b0101420 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b8600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f41b010d1c0 con 0x7f41b0102620 2024-06-03T01:09:11.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 0x7f41b019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b6600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 0x7f41b01a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 0x7f41b019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33322/0 (socket says 172.21.15.94:33322) 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5c00700 1 -- 172.21.15.94:0/430676911 learned_addr learned my addr 172.21.15.94:0/430676911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5200700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b0102620 0x7f41b019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5c00700 1 -- 172.21.15.94:0/430676911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b0102620 msgr2=0x7f41b019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5c00700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b0102620 0x7f41b019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5c00700 1 -- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 msgr2=0x7f41b01a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.828+0000 7f41b5c00700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 0x7f41b01a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41b5c00700 1 -- 172.21.15.94:0/430676911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b01a24f0 con 0x7f41b0101420 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41b5200700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b0102620 0x7f41b019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:09:11.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41b5c00700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 0x7f41b019a890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41a400efc0 tx=0x7f41a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41ae000700 1 -- 172.21.15.94:0/430676911 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40092e0 con 0x7f41b0101420 2024-06-03T01:09:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b01a2780 con 0x7f41b0101420 2024-06-03T01:09:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41ae000700 1 -- 172.21.15.94:0/430676911 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4003a00 con 0x7f41b0101420 2024-06-03T01:09:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41ae000700 1 -- 172.21.15.94:0/430676911 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4004680 con 0x7f41b0101420 2024-06-03T01:09:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.829+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b01a2cc0 con 0x7f41b0101420 2024-06-03T01:09:11.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.831+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b0069180 con 0x7f41b0101420 2024-06-03T01:09:11.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.833+0000 7f41ae000700 1 -- 172.21.15.94:0/430676911 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41a401b030 con 0x7f41b0101420 2024-06-03T01:09:11.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.834+0000 7f41ae000700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f41980a2c20 0x7f41980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:11.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.834+0000 7f41ae000700 1 -- 172.21.15.94:0/430676911 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f41a40cf370 con 0x7f41b0101420 2024-06-03T01:09:11.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.834+0000 7f41b5200700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f41980a2c20 0x7f41980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:11.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.835+0000 7f41b5200700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f41980a2c20 0x7f41980a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f419c009cb0 tx=0x7f419c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:11.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:11.838+0000 7f41ae000700 1 -- 172.21.15.94:0/430676911 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f41a4022020 con 0x7f41b0101420 2024-06-03T01:09:12.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.083+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f41b00feaa0 con 0x7f41980a2c20 2024-06-03T01:09:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.085+0000 7f41ae000700 1 -- 172.21.15.94:0/430676911 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f41b00feaa0 con 0x7f41980a2c20 2024-06-03T01:09:12.085 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-06-03T01:09:12.085 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-06-03T01:09:12.086 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-06-03T01:09:12.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f41980a2c20 msgr2=0x7f41980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:12.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f41980a2c20 0x7f41980a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f419c009cb0 tx=0x7f419c009260 comp rx=0 tx=0).stop 2024-06-03T01:09:12.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 msgr2=0x7f41b019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:12.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 0x7f41b019a890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41a400efc0 tx=0x7f41a400c3a0 comp rx=0 tx=0).stop 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 shutdown_connections 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b0101420 0x7f41b019a890 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f41980a2c20 0x7f41980a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41b0102fc0 0x7f41b01a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 --2- 172.21.15.94:0/430676911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b0102620 0x7f41b019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.088+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 >> 172.21.15.94:0/430676911 conn(0x7f41b00fc9e0 msgr2=0x7f41b00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.089+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 shutdown_connections 2024-06-03T01:09:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:12.089+0000 7f41b8600700 1 -- 172.21.15.94:0/430676911 wait complete. 2024-06-03T01:09:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:12 smithi107 ceph-mon[36622]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:12 smithi107 ceph-mon[36622]: pgmap v469: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:12.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:12 smithi094 ceph-mon[32119]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:12.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:12 smithi094 ceph-mon[32119]: pgmap v469: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:12 smithi179 ceph-mon[37041]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:12 smithi179 ceph-mon[37041]: pgmap v469: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:13.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-03T01:09:13.565 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:14 smithi107 ceph-mon[36622]: pgmap v470: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:14.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:14 smithi094 ceph-mon[32119]: pgmap v470: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:14.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:14 smithi179 ceph-mon[37041]: pgmap v470: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:15.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 -- 172.21.15.94:0/2271361080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618074ef0 msgr2=0x7f6618075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 --2- 172.21.15.94:0/2271361080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618074ef0 0x7f6618075370 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66040098f0 tx=0x7f6604009c00 comp rx=0 tx=0).stop 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 -- 172.21.15.94:0/2271361080 shutdown_connections 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 --2- 172.21.15.94:0/2271361080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f661810a3c0 0x7f661810c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 --2- 172.21.15.94:0/2271361080 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180764c0 0x7f6618074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 --2- 172.21.15.94:0/2271361080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618074ef0 0x7f6618075370 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 -- 172.21.15.94:0/2271361080 >> 172.21.15.94:0/2271361080 conn(0x7f66180fda90 msgr2=0x7f66180ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 -- 172.21.15.94:0/2271361080 shutdown_connections 2024-06-03T01:09:15.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.981+0000 7f661f200700 1 -- 172.21.15.94:0/2271361080 wait complete. 2024-06-03T01:09:15.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.983+0000 7f661f200700 1 Processor -- start 2024-06-03T01:09:15.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.983+0000 7f661f200700 1 -- start start 2024-06-03T01:09:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f661f200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6618074ef0 0x7f661819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f661f200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180764c0 0x7f661819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f661f200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f661810a3c0 0x7f66181a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f661f200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6618110320 con 0x7f66180764c0 2024-06-03T01:09:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f661f200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66181101a0 con 0x7f661810a3c0 2024-06-03T01:09:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f661f200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66181104a0 con 0x7f6618074ef0 2024-06-03T01:09:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f6617e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6618074ef0 0x7f661819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f6617400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180764c0 0x7f661819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f6617e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6618074ef0 0x7f661819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:37576/0 (socket says 172.21.15.94:37576) 2024-06-03T01:09:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f6617e00700 1 -- 172.21.15.94:0/2878871085 learned_addr learned my addr 172.21.15.94:0/2878871085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:15.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.984+0000 7f661d200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f661810a3c0 0x7f66181a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f661d200700 1 -- 172.21.15.94:0/2878871085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6618074ef0 msgr2=0x7f661819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f661d200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6618074ef0 0x7f661819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f661d200700 1 -- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180764c0 msgr2=0x7f661819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f661d200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180764c0 0x7f661819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f661d200700 1 -- 172.21.15.94:0/2878871085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66181a2640 con 0x7f661810a3c0 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f6617e00700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6618074ef0 0x7f661819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f661d200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f661810a3c0 0x7f66181a1fa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6610009cd0 tx=0x7f661000c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:15.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f6614c00700 1 -- 172.21.15.94:0/2878871085 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661000cd10 con 0x7f661810a3c0 2024-06-03T01:09:15.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f6614c00700 1 -- 172.21.15.94:0/2878871085 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6610003a00 con 0x7f661810a3c0 2024-06-03T01:09:15.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.985+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66181a28d0 con 0x7f661810a3c0 2024-06-03T01:09:15.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.986+0000 7f6614c00700 1 -- 172.21.15.94:0/2878871085 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66100045a0 con 0x7f661810a3c0 2024-06-03T01:09:15.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.986+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66181a2e10 con 0x7f661810a3c0 2024-06-03T01:09:15.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.987+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6618069180 con 0x7f661810a3c0 2024-06-03T01:09:15.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.989+0000 7f6614c00700 1 -- 172.21.15.94:0/2878871085 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f661001a020 con 0x7f661810a3c0 2024-06-03T01:09:15.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.990+0000 7f6614c00700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f65fc0a2ca0 0x7f65fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:15.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.991+0000 7f6614c00700 1 -- 172.21.15.94:0/2878871085 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f66100cf2f0 con 0x7f661810a3c0 2024-06-03T01:09:15.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.991+0000 7f6617e00700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f65fc0a2ca0 0x7f65fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:15.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.991+0000 7f6617e00700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f65fc0a2ca0 0x7f65fc0a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6608005950 tx=0x7f6608009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:15.993+0000 7f6614c00700 1 -- 172.21.15.94:0/2878871085 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6610010040 con 0x7f661810a3c0 2024-06-03T01:09:16.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.226+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6618062990 con 0x7f65fc0a2ca0 2024-06-03T01:09:16.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.244+0000 7f6614c00700 1 -- 172.21.15.94:0/2878871085 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6618062990 con 0x7f65fc0a2ca0 2024-06-03T01:09:16.244 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.a smithi094 running (4m) 2m ago 4m 14.7M - 19.0.0-2946-g3d89accf b045ac545f82 9f54f277a773 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.b smithi107 running (4m) 2m ago 4m 25.4M - 19.0.0-2946-g3d89accf b045ac545f82 ab857863a9e0 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.c smithi179 running (5m) 2m ago 5m 18.8M - 19.0.0-2946-g3d89accf b045ac545f82 18ef3204d404 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.d smithi094 running (5m) 2m ago 5m 13.0M - 19.0.0-2946-g3d89accf b045ac545f82 95656a1bf6c1 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.e smithi107 running (5m) 2m ago 5m 17.5M - 19.0.0-2946-g3d89accf b045ac545f82 b23cdaba92a4 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.f smithi179 running (5m) 2m ago 5m 17.6M - 19.0.0-2946-g3d89accf b045ac545f82 11b6c45346f1 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.g smithi094 running (4m) 2m ago 4m 14.9M - 19.0.0-2946-g3d89accf b045ac545f82 3daa0df28bb2 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.h smithi107 running (4m) 2m ago 4m 18.6M - 19.0.0-2946-g3d89accf b045ac545f82 a5069b673266 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.i smithi179 running (4m) 2m ago 4m 19.1M - 19.0.0-2946-g3d89accf b045ac545f82 e0f7f4e23e85 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.j smithi094 running (5m) 2m ago 5m 13.0M - 19.0.0-2946-g3d89accf b045ac545f82 85d667e7ee53 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.k smithi107 running (5m) 2m ago 5m 16.3M - 19.0.0-2946-g3d89accf b045ac545f82 fb8529a3db0b 2024-06-03T01:09:16.245 INFO:teuthology.orchestra.run.smithi094.stdout:mds.l smithi179 running (5m) 2m ago 5m 18.8M - 19.0.0-2946-g3d89accf b045ac545f82 c7fcb93dc625 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.x smithi094 *:9283,8765 running (13m) 2m ago 13m 516M - 19.0.0-2946-g3d89accf b045ac545f82 0dcafd448720 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.y smithi107 *:8443,8765 running (11m) 2m ago 11m 454M - 19.0.0-2946-g3d89accf b045ac545f82 726312a8f79e 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.z smithi179 *:8443,8765 running (11m) 2m ago 11m 456M - 19.0.0-2946-g3d89accf b045ac545f82 729cadb4f365 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a smithi094 running (13m) 2m ago 13m 49.2M 2048M 19.0.0-2946-g3d89accf b045ac545f82 a6a853b7a1fe 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b smithi107 running (11m) 2m ago 11m 45.4M 2048M 19.0.0-2946-g3d89accf b045ac545f82 62de82cd24ed 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c smithi179 running (11m) 2m ago 11m 42.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 a77623c6a7da 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (10m) 2m ago 10m 54.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d4caab23f6a9 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi107 running (10m) 2m ago 10m 47.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d303bf69fedf 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi179 running (10m) 2m ago 10m 50.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 3e8d9f9aa3a7 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi094 running (9m) 2m ago 9m 51.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d30a9d221160 2024-06-03T01:09:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi107 running (9m) 2m ago 9m 54.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 fe676de71df3 2024-06-03T01:09:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi179 running (8m) 2m ago 8m 55.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 1b83988e1ece 2024-06-03T01:09:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 smithi094 running (8m) 2m ago 8m 49.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 8986606ba81e 2024-06-03T01:09:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 smithi107 running (7m) 2m ago 7m 51.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 f16060dfaa7d 2024-06-03T01:09:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 smithi179 running (7m) 2m ago 7m 54.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 16697b8af8b0 2024-06-03T01:09:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 smithi094 running (6m) 2m ago 6m 48.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 8ba9a60ff7dc 2024-06-03T01:09:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 smithi107 running (6m) 2m ago 6m 56.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 653c01c54942 2024-06-03T01:09:16.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 smithi179 running (5m) 2m ago 5m 49.3M 4272M 19.0.0-2946-g3d89accf b045ac545f82 468a060ffddc 2024-06-03T01:09:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f65fc0a2ca0 msgr2=0x7f65fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f65fc0a2ca0 0x7f65fc0a5160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6608005950 tx=0x7f6608009260 comp rx=0 tx=0).stop 2024-06-03T01:09:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f661810a3c0 msgr2=0x7f66181a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f661810a3c0 0x7f66181a1fa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6610009cd0 tx=0x7f661000c620 comp rx=0 tx=0).stop 2024-06-03T01:09:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 shutdown_connections 2024-06-03T01:09:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f661810a3c0 0x7f66181a1fa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:16.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f65fc0a2ca0 0x7f65fc0a5160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6618074ef0 0x7f661819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 --2- 172.21.15.94:0/2878871085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66180764c0 0x7f661819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 >> 172.21.15.94:0/2878871085 conn(0x7f66180fda90 msgr2=0x7f66181062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.247+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 shutdown_connections 2024-06-03T01:09:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:16.248+0000 7f661f200700 1 -- 172.21.15.94:0/2878871085 wait complete. 2024-06-03T01:09:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:16 smithi094 ceph-mon[32119]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:16 smithi094 ceph-mon[32119]: pgmap v471: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:16 smithi107 ceph-mon[36622]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:16 smithi107 ceph-mon[36622]: pgmap v471: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:16 smithi179 ceph-mon[37041]: from='client.25055 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:16 smithi179 ceph-mon[37041]: pgmap v471: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:17.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-03T01:09:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:18 smithi107 ceph-mon[36622]: pgmap v472: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:18.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:18 smithi094 ceph-mon[32119]: pgmap v472: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:18.854 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:18 smithi179 ceph-mon[37041]: pgmap v472: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:20.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 -- 172.21.15.94:0/2427350069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14105910 msgr2=0x7ffb14105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 --2- 172.21.15.94:0/2427350069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14105910 0x7ffb14105d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb000099c0 tx=0x7ffb00009cd0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 -- 172.21.15.94:0/2427350069 shutdown_connections 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 --2- 172.21.15.94:0/2427350069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb141062b0 0x7ffb1410cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 --2- 172.21.15.94:0/2427350069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14104710 0x7ffb14104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 --2- 172.21.15.94:0/2427350069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb14105910 0x7ffb14105d70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 -- 172.21.15.94:0/2427350069 >> 172.21.15.94:0/2427350069 conn(0x7ffb140ffcb0 msgr2=0x7ffb141020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 -- 172.21.15.94:0/2427350069 shutdown_connections 2024-06-03T01:09:20.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.237+0000 7ffb1a600700 1 -- 172.21.15.94:0/2427350069 wait complete. 2024-06-03T01:09:20.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.238+0000 7ffb1a600700 1 Processor -- start 2024-06-03T01:09:20.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.238+0000 7ffb1a600700 1 -- start start 2024-06-03T01:09:20.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb1a600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb14104710 0x7ffb1419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:20.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb1a600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14105910 0x7ffb1419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb1a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb141062b0 0x7ffb141a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb1a600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141102f0 con 0x7ffb14104710 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb1a600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14110170 con 0x7ffb141062b0 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb1a600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14110470 con 0x7ffb14105910 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb13400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb14104710 0x7ffb1419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb13400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb14104710 0x7ffb1419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59876/0 (socket says 172.21.15.94:59876) 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb13400700 1 -- 172.21.15.94:0/1503378516 learned_addr learned my addr 172.21.15.94:0/1503378516 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb13e00700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb141062b0 0x7ffb141a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:20.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.239+0000 7ffb12a00700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14105910 0x7ffb1419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb13400700 1 -- 172.21.15.94:0/1503378516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14105910 msgr2=0x7ffb1419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb13400700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14105910 0x7ffb1419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb13400700 1 -- 172.21.15.94:0/1503378516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb141062b0 msgr2=0x7ffb141a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb13400700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb141062b0 0x7ffb141a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb13400700 1 -- 172.21.15.94:0/1503378516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb141a2530 con 0x7ffb14104710 2024-06-03T01:09:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb13400700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb14104710 0x7ffb1419a8d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400e980 tx=0x7ffb0400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb0be00700 1 -- 172.21.15.94:0/1503378516 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04004230 con 0x7ffb14104710 2024-06-03T01:09:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb0be00700 1 -- 172.21.15.94:0/1503378516 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb040043d0 con 0x7ffb14104710 2024-06-03T01:09:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb141a2820 con 0x7ffb14104710 2024-06-03T01:09:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.240+0000 7ffb0be00700 1 -- 172.21.15.94:0/1503378516 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04003da0 con 0x7ffb14104710 2024-06-03T01:09:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.241+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141a2d10 con 0x7ffb14104710 2024-06-03T01:09:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.242+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb14069180 con 0x7ffb14104710 2024-06-03T01:09:20.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.244+0000 7ffb0be00700 1 -- 172.21.15.94:0/1503378516 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb0400c7c0 con 0x7ffb14104710 2024-06-03T01:09:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.245+0000 7ffb0be00700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ffaf80a2c20 0x7ffaf80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.245+0000 7ffb0be00700 1 -- 172.21.15.94:0/1503378516 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7ffb040cf620 con 0x7ffb14104710 2024-06-03T01:09:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.245+0000 7ffb12a00700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ffaf80a2c20 0x7ffaf80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:20.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.246+0000 7ffb12a00700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ffaf80a2c20 0x7ffaf80a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb00000c00 tx=0x7ffb00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:20.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.249+0000 7ffb0be00700 1 -- 172.21.15.94:0/1503378516 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffb0409c1a0 con 0x7ffb14104710 2024-06-03T01:09:20.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.491+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ffb140628d0 con 0x7ffaf80a2c20 2024-06-03T01:09:20.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.507+0000 7ffb0be00700 1 -- 172.21.15.94:0/1503378516 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ffb140628d0 con 0x7ffaf80a2c20 2024-06-03T01:09:20.507 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-03T01:09:20.507 INFO:teuthology.orchestra.run.smithi094.stdout:mds.all 12/12 2m ago 5m smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-06-03T01:09:20.507 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 3/3 2m ago 11m smithi094=x;smithi107=y;smithi179=z;count:3 2024-06-03T01:09:20.507 INFO:teuthology.orchestra.run.smithi094.stdout:mon 3/3 2m ago 11m smithi094:172.21.15.94=a;smithi107:172.21.15.107=b;smithi179:172.21.15.179=c;count:3 2024-06-03T01:09:20.507 INFO:teuthology.orchestra.run.smithi094.stdout:osd 12 2m ago - 2024-06-03T01:09:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.510+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ffaf80a2c20 msgr2=0x7ffaf80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ffaf80a2c20 0x7ffaf80a50e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb00000c00 tx=0x7ffb00005fb0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb14104710 msgr2=0x7ffb1419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb14104710 0x7ffb1419a8d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400e980 tx=0x7ffb0400ec90 comp rx=0 tx=0).stop 2024-06-03T01:09:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 shutdown_connections 2024-06-03T01:09:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb141062b0 0x7ffb141a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7ffaf80a2c20 0x7ffaf80a50e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb14104710 0x7ffb1419a8d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 --2- 172.21.15.94:0/1503378516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb14105910 0x7ffb1419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:20.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 >> 172.21.15.94:0/1503378516 conn(0x7ffb140ffcb0 msgr2=0x7ffb14108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:20.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 shutdown_connections 2024-06-03T01:09:20.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:20.511+0000 7ffb1a600700 1 -- 172.21.15.94:0/1503378516 wait complete. 2024-06-03T01:09:20.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:20 smithi107 ceph-mon[36622]: pgmap v473: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:20 smithi094 ceph-mon[32119]: pgmap v473: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:20 smithi179 ceph-mon[37041]: pgmap v473: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:21.571 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-03T01:09:21.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:21 smithi107 ceph-mon[36622]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:21.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:21 smithi094 ceph-mon[32119]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:21.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:21 smithi179 ceph-mon[37041]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:21.996 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:22.756 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:22 smithi179 ceph-mon[37041]: pgmap v474: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:22 smithi107 ceph-mon[36622]: pgmap v474: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:22.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:22 smithi094 ceph-mon[32119]: pgmap v474: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 -- 172.21.15.94:0/41823993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388104740 msgr2=0x7f3388104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 --2- 172.21.15.94:0/41823993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388104740 0x7f3388104b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f337c0099c0 tx=0x7f337c009cd0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 -- 172.21.15.94:0/41823993 shutdown_connections 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 --2- 172.21.15.94:0/41823993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33881062e0 0x7f338810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 --2- 172.21.15.94:0/41823993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388104740 0x7f3388104b60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 --2- 172.21.15.94:0/41823993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388105940 0x7f3388105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 -- 172.21.15.94:0/41823993 >> 172.21.15.94:0/41823993 conn(0x7f33880ffce0 msgr2=0x7f3388102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 -- 172.21.15.94:0/41823993 shutdown_connections 2024-06-03T01:09:24.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.440+0000 7f3390200700 1 -- 172.21.15.94:0/41823993 wait complete. 2024-06-03T01:09:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.441+0000 7f3390200700 1 Processor -- start 2024-06-03T01:09:24.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f3390200700 1 -- start start 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f3390200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388104740 0x7f338819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f3390200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388105940 0x7f338819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f3390200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33881062e0 0x7f33881a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f3390200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3388110370 con 0x7f3388105940 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f3390200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33881101f0 con 0x7f3388104740 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f3390200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33881104f0 con 0x7f33881062e0 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f338d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388104740 0x7f338819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f338d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388104740 0x7f338819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57482/0 (socket says 172.21.15.94:57482) 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.442+0000 7f338d800700 1 -- 172.21.15.94:0/4044524815 learned_addr learned my addr 172.21.15.94:0/4044524815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338ce00700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388105940 0x7f338819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:24.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338e200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33881062e0 0x7f33881a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338d800700 1 -- 172.21.15.94:0/4044524815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33881062e0 msgr2=0x7f33881a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338d800700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33881062e0 0x7f33881a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338d800700 1 -- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388105940 msgr2=0x7f338819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338d800700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388105940 0x7f338819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338d800700 1 -- 172.21.15.94:0/4044524815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33881a2610 con 0x7f3388104740 2024-06-03T01:09:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338e200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33881062e0 0x7f33881a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:09:24.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338ce00700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388105940 0x7f338819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:09:24.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.443+0000 7f338d800700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388104740 0x7f338819a9b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f337c0098e0 tx=0x7f337c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:24.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.444+0000 7f3386000700 1 -- 172.21.15.94:0/4044524815 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c027070 con 0x7f3388104740 2024-06-03T01:09:24.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.444+0000 7f3386000700 1 -- 172.21.15.94:0/4044524815 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c00fbc0 con 0x7f3388104740 2024-06-03T01:09:24.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.444+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33881a28a0 con 0x7f3388104740 2024-06-03T01:09:24.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.444+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33881a2d50 con 0x7f3388104740 2024-06-03T01:09:24.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.444+0000 7f3386000700 1 -- 172.21.15.94:0/4044524815 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c017650 con 0x7f3388104740 2024-06-03T01:09:24.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.445+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3368005330 con 0x7f3388104740 2024-06-03T01:09:24.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.447+0000 7f3386000700 1 -- 172.21.15.94:0/4044524815 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f337c02a070 con 0x7f3388104740 2024-06-03T01:09:24.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.448+0000 7f3386000700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f33700a2ca0 0x7f33700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:24.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.448+0000 7f3386000700 1 -- 172.21.15.94:0/4044524815 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f337c0d01d0 con 0x7f3388104740 2024-06-03T01:09:24.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.448+0000 7f338ce00700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f33700a2ca0 0x7f33700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:24.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.449+0000 7f338ce00700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f33700a2ca0 0x7f33700a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3374009cb0 tx=0x7f3374009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:24.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.452+0000 7f3386000700 1 -- 172.21.15.94:0/4044524815 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f337c05e850 con 0x7f3388104740 2024-06-03T01:09:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:24 smithi094 ceph-mon[32119]: pgmap v475: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:24.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.682+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3368000bc0 con 0x7f33700a2ca0 2024-06-03T01:09:24.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.685+0000 7f3386000700 1 -- 172.21.15.94:0/4044524815 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3368000bc0 con 0x7f33700a2ca0 2024-06-03T01:09:24.685 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-06-03T01:09:24.685 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-06-03T01:09:24.685 INFO:teuthology.orchestra.run.smithi094.stdout:smithi107 172.21.15.107 2024-06-03T01:09:24.685 INFO:teuthology.orchestra.run.smithi094.stdout:smithi179 172.21.15.179 2024-06-03T01:09:24.685 INFO:teuthology.orchestra.run.smithi094.stdout:3 hosts in cluster 2024-06-03T01:09:24.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f33700a2ca0 msgr2=0x7f33700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:24.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f33700a2ca0 0x7f33700a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3374009cb0 tx=0x7f3374009260 comp rx=0 tx=0).stop 2024-06-03T01:09:24.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388104740 msgr2=0x7f338819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388104740 0x7f338819a9b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f337c0098e0 tx=0x7f337c009580 comp rx=0 tx=0).stop 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 shutdown_connections 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388104740 0x7f338819a9b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f33700a2ca0 0x7f33700a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33881062e0 0x7f33881a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 --2- 172.21.15.94:0/4044524815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3388105940 0x7f338819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 >> 172.21.15.94:0/4044524815 conn(0x7f33880ffce0 msgr2=0x7f3388108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 shutdown_connections 2024-06-03T01:09:24.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:24.688+0000 7f3390200700 1 -- 172.21.15.94:0/4044524815 wait complete. 2024-06-03T01:09:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:24 smithi107 ceph-mon[36622]: pgmap v475: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:24 smithi179 ceph-mon[37041]: pgmap v475: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:25.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:25 smithi094 ceph-mon[32119]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:25.699 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-03T01:09:25.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:25 smithi107 ceph-mon[36622]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:25.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:25 smithi179 ceph-mon[37041]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:26.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:26 smithi094 ceph-mon[32119]: pgmap v476: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:26.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:26 smithi107 ceph-mon[36622]: pgmap v476: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:26 smithi179 ceph-mon[37041]: pgmap v476: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:28.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 -- 172.21.15.94:0/3300228316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad8104090 msgr2=0x7f6ad810a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 --2- 172.21.15.94:0/3300228316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad8104090 0x7f6ad810a940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009fc0 tx=0x7f6ad000b330 comp rx=0 tx=0).stop 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 -- 172.21.15.94:0/3300228316 shutdown_connections 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 --2- 172.21.15.94:0/3300228316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad8104090 0x7f6ad810a940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 --2- 172.21.15.94:0/3300228316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 0x7f6ad8102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 --2- 172.21.15.94:0/3300228316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ad81036f0 0x7f6ad8103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 -- 172.21.15.94:0/3300228316 >> 172.21.15.94:0/3300228316 conn(0x7f6ad80fda90 msgr2=0x7f6ad80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 -- 172.21.15.94:0/3300228316 shutdown_connections 2024-06-03T01:09:28.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.586+0000 7f6ae0000700 1 -- 172.21.15.94:0/3300228316 wait complete. 2024-06-03T01:09:28.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.587+0000 7f6ae0000700 1 Processor -- start 2024-06-03T01:09:28.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.587+0000 7f6ae0000700 1 -- start start 2024-06-03T01:09:28.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.587+0000 7f6ae0000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 0x7f6ad8076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6ae0000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad81036f0 0x7f6ad8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6ae0000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ad8104090 0x7f6ad8074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6ae0000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810e110 con 0x7f6ad81024f0 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6ae0000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810df90 con 0x7f6ad81036f0 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6ae0000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad810e290 con 0x7f6ad8104090 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 0x7f6ad8076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 0x7f6ad8076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38062/0 (socket says 172.21.15.94:38062) 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 -- 172.21.15.94:0/1458572946 learned_addr learned my addr 172.21.15.94:0/1458572946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6adcc00700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad81036f0 0x7f6ad8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6ade000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ad8104090 0x7f6ad8074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 -- 172.21.15.94:0/1458572946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ad8104090 msgr2=0x7f6ad8074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ad8104090 0x7f6ad8074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 -- 172.21.15.94:0/1458572946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad81036f0 msgr2=0x7f6ad8074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:28.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad81036f0 0x7f6ad8074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.588+0000 7f6add600700 1 -- 172.21.15.94:0/1458572946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad8075670 con 0x7f6ad81024f0 2024-06-03T01:09:28.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.589+0000 7f6add600700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 0x7f6ad8076230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00e9f0 tx=0x7f6acc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:28.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.589+0000 7f6ad6000700 1 -- 172.21.15.94:0/1458572946 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc004220 con 0x7f6ad81024f0 2024-06-03T01:09:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.589+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad81a2850 con 0x7f6ad81024f0 2024-06-03T01:09:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.589+0000 7f6ad6000700 1 -- 172.21.15.94:0/1458572946 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6acc0043c0 con 0x7f6ad81024f0 2024-06-03T01:09:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.589+0000 7f6ad6000700 1 -- 172.21.15.94:0/1458572946 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc003ce0 con 0x7f6ad81024f0 2024-06-03T01:09:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.589+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad81a2d90 con 0x7f6ad81024f0 2024-06-03T01:09:28.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.590+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad8069180 con 0x7f6ad81024f0 2024-06-03T01:09:28.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.592+0000 7f6ad6000700 1 -- 172.21.15.94:0/1458572946 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6acc00c7c0 con 0x7f6ad81024f0 2024-06-03T01:09:28.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.594+0000 7f6ad6000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ac00a2c20 0x7f6ac00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:28.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.594+0000 7f6ad6000700 1 -- 172.21.15.94:0/1458572946 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6acc0cf540 con 0x7f6ad81024f0 2024-06-03T01:09:28.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.594+0000 7f6adcc00700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ac00a2c20 0x7f6ac00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:28.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.595+0000 7f6adcc00700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ac00a2c20 0x7f6ac00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009c60 tx=0x7f6ac4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:28.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.598+0000 7f6ad6000700 1 -- 172.21.15.94:0/1458572946 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6acc09c0c0 con 0x7f6ad81024f0 2024-06-03T01:09:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:28 smithi107 ceph-mon[36622]: pgmap v477: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:28.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.831+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 --> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6ad80628d0 con 0x7f6ac00a2c20 2024-06-03T01:09:28.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:28 smithi094 ceph-mon[32119]: pgmap v477: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:28.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.834+0000 7f6ad6000700 1 -- 172.21.15.94:0/1458572946 <== mgr.14152 v2:172.21.15.94:6800/4120992823 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6ad80628d0 con 0x7f6ac00a2c20 2024-06-03T01:09:28.834 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-03T01:09:28.834 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 6m ago Has a FileSystem, LVM detected 2024-06-03T01:09:28.834 INFO:teuthology.orchestra.run.smithi094.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-03T01:09:28.835 INFO:teuthology.orchestra.run.smithi094.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-03T01:09:28.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ac00a2c20 msgr2=0x7f6ac00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:28.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ac00a2c20 0x7f6ac00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009c60 tx=0x7f6ac4009260 comp rx=0 tx=0).stop 2024-06-03T01:09:28.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 msgr2=0x7f6ad8076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:28.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 0x7f6ad8076230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00e9f0 tx=0x7f6acc00ed00 comp rx=0 tx=0).stop 2024-06-03T01:09:28.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 shutdown_connections 2024-06-03T01:09:28.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6ac00a2c20 0x7f6ac00a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ad8104090 0x7f6ad8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad81024f0 0x7f6ad8076230 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 --2- 172.21.15.94:0/1458572946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ad81036f0 0x7f6ad8074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:28.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 >> 172.21.15.94:0/1458572946 conn(0x7f6ad80fda90 msgr2=0x7f6ad81069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:28.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 shutdown_connections 2024-06-03T01:09:28.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:28.837+0000 7f6ae0000700 1 -- 172.21.15.94:0/1458572946 wait complete. 2024-06-03T01:09:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:28 smithi179 ceph-mon[37041]: pgmap v477: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:29 smithi094 ceph-mon[32119]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:29 smithi107 ceph-mon[36622]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:29.825 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-03T01:09:29.834 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:09:29.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-03T01:09:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:29 smithi179 ceph-mon[37041]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:09:30.277 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:30.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:30 smithi107 ceph-mon[36622]: pgmap v478: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:30.830 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:30 smithi094 ceph-mon[32119]: pgmap v478: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:30 smithi179 ceph-mon[37041]: pgmap v478: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:32.513 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:32 smithi094 ceph-mon[32119]: pgmap v479: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:32.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:32 smithi107 ceph-mon[36622]: pgmap v479: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:32.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.808+0000 7f24fae00700 1 -- 172.21.15.94:0/3355817803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 msgr2=0x7f24f4104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:32.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.808+0000 7f24fae00700 1 --2- 172.21.15.94:0/3355817803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f4104b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24e40099c0 tx=0x7f24e4009cd0 comp rx=0 tx=0).stop 2024-06-03T01:09:32.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.809+0000 7f24fae00700 1 -- 172.21.15.94:0/3355817803 shutdown_connections 2024-06-03T01:09:32.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.809+0000 7f24fae00700 1 --2- 172.21.15.94:0/3355817803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24f41062a0 0x7f24f410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:32.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.809+0000 7f24fae00700 1 --2- 172.21.15.94:0/3355817803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f4104b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:32.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.809+0000 7f24fae00700 1 --2- 172.21.15.94:0/3355817803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24f4105900 0x7f24f4105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:32.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.809+0000 7f24fae00700 1 -- 172.21.15.94:0/3355817803 >> 172.21.15.94:0/3355817803 conn(0x7f24f40ffca0 msgr2=0x7f24f41020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:32.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.809+0000 7f24fae00700 1 -- 172.21.15.94:0/3355817803 shutdown_connections 2024-06-03T01:09:32.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.809+0000 7f24fae00700 1 -- 172.21.15.94:0/3355817803 wait complete. 2024-06-03T01:09:32.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 Processor -- start 2024-06-03T01:09:32.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 -- start start 2024-06-03T01:09:32.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:32.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24f4105900 0x7f24f419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:32.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24f41062a0 0x7f24f41a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:32.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4110310 con 0x7f24f4104700 2024-06-03T01:09:32.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4110190 con 0x7f24f41062a0 2024-06-03T01:09:32.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.810+0000 7f24fae00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4110490 con 0x7f24f4105900 2024-06-03T01:09:32.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:32.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f419a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38074/0 (socket says 172.21.15.94:38074) 2024-06-03T01:09:32.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 -- 172.21.15.94:0/1471389305 learned_addr learned my addr 172.21.15.94:0/1471389305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:32.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3400700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24f4105900 0x7f24f419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:32.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f8e00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24f41062a0 0x7f24f41a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:32.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 -- 172.21.15.94:0/1471389305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24f4105900 msgr2=0x7f24f419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:32.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24f4105900 0x7f24f419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 -- 172.21.15.94:0/1471389305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24f41062a0 msgr2=0x7f24f41a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24f41062a0 0x7f24f41a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.811+0000 7f24f3e00700 1 -- 172.21.15.94:0/1471389305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f41a25e0 con 0x7f24f4104700 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.812+0000 7f24f3e00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f419a980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24e400bee0 tx=0x7f24e400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.812+0000 7f24f0c00700 1 -- 172.21.15.94:0/1471389305 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4009e60 con 0x7f24f4104700 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.812+0000 7f24f0c00700 1 -- 172.21.15.94:0/1471389305 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e4003e30 con 0x7f24f4104700 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.812+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f41a2870 con 0x7f24f4104700 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.812+0000 7f24f0c00700 1 -- 172.21.15.94:0/1471389305 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e40173f0 con 0x7f24f4104700 2024-06-03T01:09:32.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.812+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f41a2d20 con 0x7f24f4104700 2024-06-03T01:09:32.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.813+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f4069180 con 0x7f24f4104700 2024-06-03T01:09:32.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.815+0000 7f24f0c00700 1 -- 172.21.15.94:0/1471389305 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24e400f660 con 0x7f24f4104700 2024-06-03T01:09:32.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.816+0000 7f24f0c00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24d80a2ca0 0x7f24d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:32.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.817+0000 7f24f0c00700 1 -- 172.21.15.94:0/1471389305 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f24e40d0100 con 0x7f24f4104700 2024-06-03T01:09:32.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.817+0000 7f24f3400700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24d80a2ca0 0x7f24d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:32.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.818+0000 7f24f3400700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24d80a2ca0 0x7f24d80a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009c60 tx=0x7f24e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:32.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:32.821+0000 7f24f0c00700 1 -- 172.21.15.94:0/1471389305 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f24e4013070 con 0x7f24f4104700 2024-06-03T01:09:32.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:32 smithi179 ceph-mon[37041]: pgmap v479: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:33.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.095+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f24f40630a0 con 0x7f24f4104700 2024-06-03T01:09:33.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.096+0000 7f24f0c00700 1 -- 172.21.15.94:0/1471389305 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3100 (secure 0 0 0) 0x7f24e409cd10 con 0x7f24f4104700 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:e27 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 27 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-06-03T01:04:57.629349+0000 2024-06-03T01:09:33.097 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-06-03T01:06:22.586009+0000 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 300 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 5 2024-06-03T01:09:33.098 INFO:teuthology.orchestra.run.smithi094.stdout:in 0,1,2,3,4 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24524,1=24505,2=24490,3=24485,4=24478} 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3,4] 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 0 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 24524 members: 24478,24485,24490,24505,24524 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b{0:24524} state up:active seq 8 addr [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.099 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.e{0:24467} state up:standby-replay seq 1 addr [v2:172.21.15.107:6834/2528321857,v1:172.21.15.107:6835/2528321857] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.i{1:24505} state up:active seq 30 addr [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.l{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.179:6832/2660538909,v1:172.21.15.179:6833/2660538909] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c{2:24490} state up:active seq 31 addr [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.k{2:24437} state up:standby-replay seq 1 addr [v2:172.21.15.107:6832/1330319563,v1:172.21.15.107:6833/1330319563] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.h{3:24485} state up:active seq 31 addr [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a{3:14400} state up:standby-replay seq 1 addr [v2:172.21.15.94:6840/3437199777,v1:172.21.15.94:6841/3437199777] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.f{4:24478} state up:active seq 33 addr [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.g{4:14388} state up:standby-replay seq 1 addr [v2:172.21.15.94:6838/848723917,v1:172.21.15.94:6839/848723917] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-06-03T01:09:33.100 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:09:33.101 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.j{-1:14358} state up:standby seq 1 addr [v2:172.21.15.94:6834/3523136446,v1:172.21.15.94:6835/3523136446] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.101 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d{-1:14364} state up:standby seq 1 addr [v2:172.21.15.94:6836/3219738188,v1:172.21.15.94:6837/3219738188] compat {c=[1],r=[1],i=[1fff]}] 2024-06-03T01:09:33.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.099+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24d80a2ca0 msgr2=0x7f24d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:33.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.099+0000 7f24fae00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24d80a2ca0 0x7f24d80a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009c60 tx=0x7f24e0009260 comp rx=0 tx=0).stop 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 msgr2=0x7f24f419a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f419a980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f24e400bee0 tx=0x7f24e400bf10 comp rx=0 tx=0).stop 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 shutdown_connections 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24f41062a0 0x7f24f41a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f24d80a2ca0 0x7f24d80a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24f4104700 0x7f24f419a980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 --2- 172.21.15.94:0/1471389305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24f4105900 0x7f24f419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 >> 172.21.15.94:0/1471389305 conn(0x7f24f40ffca0 msgr2=0x7f24f4108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 shutdown_connections 2024-06-03T01:09:33.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:33.100+0000 7f24fae00700 1 -- 172.21.15.94:0/1471389305 wait complete. 2024-06-03T01:09:33.103 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-06-03T01:09:33.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:33 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1471389305' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:09:33.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:33 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1471389305' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:09:33.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:33 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1471389305' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-03T01:09:34.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-03T01:09:34.579 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:34.724 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:34 smithi094 ceph-mon[32119]: pgmap v480: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:34 smithi107 ceph-mon[36622]: pgmap v480: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:34.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:34 smithi179 ceph-mon[37041]: pgmap v480: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:36.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 -- 172.21.15.94:0/3624545799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b6810a6d0 msgr2=0x7f1b68110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/3624545799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b6810a6d0 0x7f1b68110f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b0a0 tx=0x7f1b6000b3b0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 -- 172.21.15.94:0/3624545799 shutdown_connections 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/3624545799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b6810a6d0 0x7f1b68110f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/3624545799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b68108b30 0x7f1b68108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/3624545799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b68109d30 0x7f1b6810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 -- 172.21.15.94:0/3624545799 >> 172.21.15.94:0/3624545799 conn(0x7f1b68077b00 msgr2=0x7f1b68079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 -- 172.21.15.94:0/3624545799 shutdown_connections 2024-06-03T01:09:36.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.690+0000 7f1b6cc00700 1 -- 172.21.15.94:0/3624545799 wait complete. 2024-06-03T01:09:36.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.691+0000 7f1b6cc00700 1 Processor -- start 2024-06-03T01:09:36.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.691+0000 7f1b6cc00700 1 -- start start 2024-06-03T01:09:36.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.691+0000 7f1b6cc00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b68108b30 0x7f1b6819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:36.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.691+0000 7f1b6cc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b68109d30 0x7f1b6819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:36.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b6cc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b6810a6d0 0x7f1b681a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b6cc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b68114750 con 0x7f1b68109d30 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b6cc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b681145d0 con 0x7f1b6810a6d0 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b6cc00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b681148d0 con 0x7f1b68108b30 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b65e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b68108b30 0x7f1b6819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b65e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b68108b30 0x7f1b6819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38418/0 (socket says 172.21.15.94:38418) 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b65e00700 1 -- 172.21.15.94:0/1616708365 learned_addr learned my addr 172.21.15.94:0/1616708365 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b66800700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b6810a6d0 0x7f1b681a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:36.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b65400700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b68109d30 0x7f1b6819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:36.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b66800700 1 -- 172.21.15.94:0/1616708365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b68108b30 msgr2=0x7f1b6819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:36.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b66800700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b68108b30 0x7f1b6819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b66800700 1 -- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b68109d30 msgr2=0x7f1b6819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b66800700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b68109d30 0x7f1b6819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b66800700 1 -- 172.21.15.94:0/1616708365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b681a6930 con 0x7f1b6810a6d0 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.692+0000 7f1b65e00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b68108b30 0x7f1b6819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.693+0000 7f1b66800700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b6810a6d0 0x7f1b681a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b070 tx=0x7f1b60004f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.693+0000 7f1b5ea00700 1 -- 172.21.15.94:0/1616708365 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60021070 con 0x7f1b6810a6d0 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.693+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b681a6bc0 con 0x7f1b6810a6d0 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.693+0000 7f1b5ea00700 1 -- 172.21.15.94:0/1616708365 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b60007b80 con 0x7f1b6810a6d0 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.693+0000 7f1b5ea00700 1 -- 172.21.15.94:0/1616708365 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b600052c0 con 0x7f1b6810a6d0 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.693+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b681a70c0 con 0x7f1b6810a6d0 2024-06-03T01:09:36.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.694+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b68069180 con 0x7f1b6810a6d0 2024-06-03T01:09:36.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.696+0000 7f1b5ea00700 1 -- 172.21.15.94:0/1616708365 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b60024070 con 0x7f1b6810a6d0 2024-06-03T01:09:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.698+0000 7f1b5ea00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b4c0a2c20 0x7f1b4c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.698+0000 7f1b5ea00700 1 -- 172.21.15.94:0/1616708365 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b6000f020 con 0x7f1b6810a6d0 2024-06-03T01:09:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.698+0000 7f1b65e00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b4c0a2c20 0x7f1b4c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:36.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.699+0000 7f1b65e00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b4c0a2c20 0x7f1b4c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1b6819fb90 tx=0x7f1b58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:36.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.702+0000 7f1b5ea00700 1 -- 172.21.15.94:0/1616708365 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1b6002b020 con 0x7f1b6810a6d0 2024-06-03T01:09:36.702 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:36 smithi094 ceph-mon[32119]: pgmap v481: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:36 smithi107 ceph-mon[36622]: pgmap v481: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:36 smithi179 ceph-mon[37041]: pgmap v481: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:36.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.929+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f1b680630a0 con 0x7f1b6810a6d0 2024-06-03T01:09:36.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.931+0000 7f1b5ea00700 1 -- 172.21.15.94:0/1616708365 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v100) ==== 56+0+5635 (secure 0 0 0) 0x7f1b6009d540 con 0x7f1b6810a6d0 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 100 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:fsid d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-06-03T00:55:25.857100+0000 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-06-03T01:06:35.341979+0000 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:crush_version 27 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:full_ratio 0.95 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:backfillfull_ratio 0.9 2024-06-03T01:09:36.932 INFO:teuthology.orchestra.run.smithi094.stdout:nearfull_ratio 0.85 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:require_min_compat_client luminous 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:min_compat_client luminous 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:require_osd_release squid 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:stretch_mode_enabled false 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 96 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:max_osd 12 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 up in weight 1 up_from 8 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6802/837949984,v1:172.21.15.94:6803/837949984] [v2:172.21.15.94:6804/837949984,v1:172.21.15.94:6805/837949984] exists,up 63946c12-7fdf-401c-9cbc-18db7ef2f714 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 up in weight 1 up_from 13 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/3097634475,v1:172.21.15.107:6801/3097634475] [v2:172.21.15.107:6802/3097634475,v1:172.21.15.107:6803/3097634475] exists,up 6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6800/3335947212,v1:172.21.15.179:6801/3335947212] [v2:172.21.15.179:6802/3335947212,v1:172.21.15.179:6803/3335947212] exists,up a1f9c436-0127-4d60-9cfa-4a33e22409f4 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 up in weight 1 up_from 25 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6810/667050825,v1:172.21.15.94:6811/667050825] [v2:172.21.15.94:6812/667050825,v1:172.21.15.94:6813/667050825] exists,up 43ade6dd-20a3-419d-8dba-e718f61a022e 2024-06-03T01:09:36.933 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 up in weight 1 up_from 33 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/1882504860,v1:172.21.15.107:6809/1882504860] [v2:172.21.15.107:6810/1882504860,v1:172.21.15.107:6811/1882504860] exists,up 7ed0339d-4210-4398-9092-db7bcba4048b 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 up in weight 1 up_from 39 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6808/1536083025,v1:172.21.15.179:6809/1536083025] [v2:172.21.15.179:6810/1536083025,v1:172.21.15.179:6811/1536083025] exists,up b46c6631-7947-49fd-afa2-eb39f3145766 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6818/3118996469,v1:172.21.15.94:6819/3118996469] [v2:172.21.15.94:6820/3118996469,v1:172.21.15.94:6821/3118996469] exists,up 172b5d67-f3cf-48eb-9060-0b421fb38e63 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 up in weight 1 up_from 52 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6816/2599842283,v1:172.21.15.107:6817/2599842283] [v2:172.21.15.107:6818/2599842283,v1:172.21.15.107:6819/2599842283] exists,up 05b89eaa-5143-4517-949e-e25d75b4c5b8 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 up in weight 1 up_from 60 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6816/3410192696,v1:172.21.15.179:6817/3410192696] [v2:172.21.15.179:6818/3410192696,v1:172.21.15.179:6819/3410192696] exists,up 07303dac-9a89-4c1a-8555-07075f29ed66 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 up in weight 1 up_from 68 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6826/165849117,v1:172.21.15.94:6827/165849117] [v2:172.21.15.94:6828/165849117,v1:172.21.15.94:6829/165849117] exists,up 126f498f-8208-42d7-bc3e-9dcc5f200906 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 up in weight 1 up_from 73 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/1849339133,v1:172.21.15.107:6825/1849339133] [v2:172.21.15.107:6826/1849339133,v1:172.21.15.107:6827/1849339133] exists,up 070f388a-c6f7-4438-86bc-57d8f08e4175 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 up in weight 1 up_from 81 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6824/4289321509,v1:172.21.15.179:6825/4289321509] [v2:172.21.15.179:6826/4289321509,v1:172.21.15.179:6827/4289321509] exists,up d5ee1e38-10d2-4513-9d5b-29816f8d6eb5 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.5 [8,11] 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.8 [2,11] 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.13 [4,9] 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.1d [5,9] 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.23 [4,8] 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.27 [4,6] 2024-06-03T01:09:36.934 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.29 [3,9] 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.39 [5,8] 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 4.17 [9,8] 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 4.1b [9,8,5,10] 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 4.1f [9,10] 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/127652834 expires 2024-06-04T00:56:34.111978+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/127652834 expires 2024-06-04T00:56:34.111978+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2902022794 expires 2024-06-04T00:56:34.111978+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3243526621 expires 2024-06-04T00:55:59.358088+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/2467202876 expires 2024-06-04T00:55:59.358088+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/2467202876 expires 2024-06-04T00:55:59.358088+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3121579087 expires 2024-06-04T00:55:59.358088+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/1105337314 expires 2024-06-04T00:55:59.358088+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3874005975 expires 2024-06-04T00:56:34.111978+0000 2024-06-03T01:09:36.935 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/516090952 expires 2024-06-04T00:56:34.111978+0000 2024-06-03T01:09:36.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b4c0a2c20 msgr2=0x7f1b4c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:36.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b4c0a2c20 0x7f1b4c0a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1b6819fb90 tx=0x7f1b58009260 comp rx=0 tx=0).stop 2024-06-03T01:09:36.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b6810a6d0 msgr2=0x7f1b681a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:36.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b6810a6d0 0x7f1b681a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b070 tx=0x7f1b60004f40 comp rx=0 tx=0).stop 2024-06-03T01:09:36.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 shutdown_connections 2024-06-03T01:09:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b6810a6d0 0x7f1b681a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b4c0a2c20 0x7f1b4c0a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b68108b30 0x7f1b6819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 --2- 172.21.15.94:0/1616708365 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b68109d30 0x7f1b6819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 >> 172.21.15.94:0/1616708365 conn(0x7f1b68077b00 msgr2=0x7f1b6810cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 shutdown_connections 2024-06-03T01:09:36.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:36.934+0000 7f1b6cc00700 1 -- 172.21.15.94:0/1616708365 wait complete. 2024-06-03T01:09:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:37 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1616708365' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T01:09:37.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:37 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1616708365' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T01:09:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:37 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1616708365' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-03T01:09:37.914 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-03T01:09:37.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:09:38.373 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:38.687 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:38 smithi094 ceph-mon[32119]: pgmap v482: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:38 smithi107 ceph-mon[36622]: pgmap v482: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:38 smithi179 ceph-mon[37041]: pgmap v482: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:40.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:40 smithi094 ceph-mon[32119]: pgmap v483: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:40.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:40 smithi107 ceph-mon[36622]: pgmap v483: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 -- 172.21.15.94:0/854700795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 msgr2=0x7f2370102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 --2- 172.21.15.94:0/854700795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 0x7f2370102250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f235c009920 tx=0x7f235c009c30 comp rx=0 tx=0).stop 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 -- 172.21.15.94:0/854700795 shutdown_connections 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 --2- 172.21.15.94:0/854700795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2370102790 0x7f237010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 --2- 172.21.15.94:0/854700795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23701074b0 0x7f2370107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 --2- 172.21.15.94:0/854700795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 0x7f2370102250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 -- 172.21.15.94:0/854700795 >> 172.21.15.94:0/854700795 conn(0x7f2370076640 msgr2=0x7f2370076a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 -- 172.21.15.94:0/854700795 shutdown_connections 2024-06-03T01:09:40.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.792+0000 7f2376000700 1 -- 172.21.15.94:0/854700795 wait complete. 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.793+0000 7f2376000700 1 Processor -- start 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2376000700 1 -- start start 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2376000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 0x7f237019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2376000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2370102790 0x7f237019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2376000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23701074b0 0x7f237019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2376000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2370111770 con 0x7f2370102790 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2376000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23701115f0 con 0x7f2370101dd0 2024-06-03T01:09:40.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2376000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23701118f0 con 0x7f23701074b0 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f236fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2370102790 0x7f237019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f236fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2370102790 0x7f237019b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45092/0 (socket says 172.21.15.94:45092) 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f236fe00700 1 -- 172.21.15.94:0/2785476970 learned_addr learned my addr 172.21.15.94:0/2785476970 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2374c00700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 0x7f237019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.794+0000 7f2375600700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23701074b0 0x7f237019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2375600700 1 -- 172.21.15.94:0/2785476970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 msgr2=0x7f237019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2375600700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 0x7f237019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2375600700 1 -- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2370102790 msgr2=0x7f237019b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2375600700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2370102790 0x7f237019b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2375600700 1 -- 172.21.15.94:0/2785476970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23701a0040 con 0x7f23701074b0 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f236fe00700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2370102790 0x7f237019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2374c00700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 0x7f237019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:40.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2375600700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23701074b0 0x7f237019f900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f236800e970 tx=0x7f236800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:40.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f236d600700 1 -- 172.21.15.94:0/2785476970 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004220 con 0x7f23701074b0 2024-06-03T01:09:40.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f236d600700 1 -- 172.21.15.94:0/2785476970 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2368004d10 con 0x7f23701074b0 2024-06-03T01:09:40.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23701a0330 con 0x7f23701074b0 2024-06-03T01:09:40.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.795+0000 7f236d600700 1 -- 172.21.15.94:0/2785476970 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368010700 con 0x7f23701074b0 2024-06-03T01:09:40.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.796+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23701a7bb0 con 0x7f23701074b0 2024-06-03T01:09:40.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.797+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2370069180 con 0x7f23701074b0 2024-06-03T01:09:40.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.799+0000 7f236d600700 1 -- 172.21.15.94:0/2785476970 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2368042ae0 con 0x7f23701074b0 2024-06-03T01:09:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.800+0000 7f236d600700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f23580a2c20 0x7f23580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.800+0000 7f236d600700 1 -- 172.21.15.94:0/2785476970 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f23680cf120 con 0x7f23701074b0 2024-06-03T01:09:40.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.800+0000 7f2374c00700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f23580a2c20 0x7f23580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:40.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.801+0000 7f2374c00700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f23580a2c20 0x7f23580a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2364009cd0 tx=0x7f236400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:40.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:40.802+0000 7f236d600700 1 -- 172.21.15.94:0/2785476970 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f236809bca0 con 0x7f23701074b0 2024-06-03T01:09:40.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:40 smithi179 ceph-mon[37041]: pgmap v483: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:41.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.071+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f237019c390 con 0x7f23701074b0 2024-06-03T01:09:41.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.072+0000 7f236d600700 1 -- 172.21.15.94:0/2785476970 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f236809b5a0 con 0x7f23701074b0 2024-06-03T01:09:41.073 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:09:41.073 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:06:22.586009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24478,24485,24490,24505,24524]},"id":1}]} 2024-06-03T01:09:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f23580a2c20 msgr2=0x7f23580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f23580a2c20 0x7f23580a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2364009cd0 tx=0x7f236400b040 comp rx=0 tx=0).stop 2024-06-03T01:09:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23701074b0 msgr2=0x7f237019f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:41.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23701074b0 0x7f237019f900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f236800e970 tx=0x7f236800ec80 comp rx=0 tx=0).stop 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 shutdown_connections 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2370101dd0 0x7f237019b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f23580a2c20 0x7f23580a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23701074b0 0x7f237019f900 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 --2- 172.21.15.94:0/2785476970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2370102790 0x7f237019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.075+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 >> 172.21.15.94:0/2785476970 conn(0x7f2370076640 msgr2=0x7f237010c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.076+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 shutdown_connections 2024-06-03T01:09:41.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:41.076+0000 7f2376000700 1 -- 172.21.15.94:0/2785476970 wait complete. 2024-06-03T01:09:41.077 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-06-03T01:09:41.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:41 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2785476970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:09:41.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:41 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2785476970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:09:41.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:41 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2785476970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:09:42.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:09:42.537 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:42.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:42 smithi094 ceph-mon[32119]: pgmap v484: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:42 smithi107 ceph-mon[36622]: pgmap v484: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:42.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:42 smithi179 ceph-mon[37041]: pgmap v484: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:44.671 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:44 smithi094 ceph-mon[32119]: pgmap v485: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:44.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:44 smithi107 ceph-mon[36622]: pgmap v485: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:44.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:44 smithi179 ceph-mon[37041]: pgmap v485: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.973+0000 7f66e3600700 1 -- 172.21.15.94:0/4122880217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc107840 msgr2=0x7f66dc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.973+0000 7f66e3600700 1 --2- 172.21.15.94:0/4122880217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc107840 0x7f66dc10f3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f66d400b0d0 tx=0x7f66d400b3e0 comp rx=0 tx=0).stop 2024-06-03T01:09:44.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.974+0000 7f66e3600700 1 -- 172.21.15.94:0/4122880217 shutdown_connections 2024-06-03T01:09:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.974+0000 7f66e3600700 1 --2- 172.21.15.94:0/4122880217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc107840 0x7f66dc10f3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.974+0000 7f66e3600700 1 --2- 172.21.15.94:0/4122880217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc102740 0x7f66dc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.974+0000 7f66e3600700 1 --2- 172.21.15.94:0/4122880217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66dc1030f0 0x7f66dc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.974+0000 7f66e3600700 1 -- 172.21.15.94:0/4122880217 >> 172.21.15.94:0/4122880217 conn(0x7f66dc0795f0 msgr2=0x7f66dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.974+0000 7f66e3600700 1 -- 172.21.15.94:0/4122880217 shutdown_connections 2024-06-03T01:09:44.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.974+0000 7f66e3600700 1 -- 172.21.15.94:0/4122880217 wait complete. 2024-06-03T01:09:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.975+0000 7f66e3600700 1 Processor -- start 2024-06-03T01:09:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.975+0000 7f66e3600700 1 -- start start 2024-06-03T01:09:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e3600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc102740 0x7f66dc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e3600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66dc1030f0 0x7f66dc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:44.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e3600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc107840 0x7f66dc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e3600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1127d0 con 0x7f66dc102740 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e3600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112650 con 0x7f66dc1030f0 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e3600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112950 con 0x7f66dc107840 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e0c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc102740 0x7f66dc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66dbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66dc1030f0 0x7f66dc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e0c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc102740 0x7f66dc19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34876/0 (socket says 172.21.15.94:34876) 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e0c00700 1 -- 172.21.15.94:0/1705258875 learned_addr learned my addr 172.21.15.94:0/1705258875 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:44.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e1600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc107840 0x7f66dc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:44.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66dbe00700 1 -- 172.21.15.94:0/1705258875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc107840 msgr2=0x7f66dc19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:44.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66dbe00700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc107840 0x7f66dc19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:44.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66dbe00700 1 -- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc102740 msgr2=0x7f66dc19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:44.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66dbe00700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc102740 0x7f66dc19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:44.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66dbe00700 1 -- 172.21.15.94:0/1705258875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc19fe40 con 0x7f66dc1030f0 2024-06-03T01:09:44.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.976+0000 7f66e0c00700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc102740 0x7f66dc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.977+0000 7f66dbe00700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66dc1030f0 0x7f66dc19b370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66c800e9f0 tx=0x7f66c800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.977+0000 7f66d9600700 1 -- 172.21.15.94:0/1705258875 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c800c9c0 con 0x7f66dc1030f0 2024-06-03T01:09:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.977+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc1a0130 con 0x7f66dc1030f0 2024-06-03T01:09:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.977+0000 7f66d9600700 1 -- 172.21.15.94:0/1705258875 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c800cb60 con 0x7f66dc1030f0 2024-06-03T01:09:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.977+0000 7f66d9600700 1 -- 172.21.15.94:0/1705258875 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8018680 con 0x7f66dc1030f0 2024-06-03T01:09:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.977+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc1a79b0 con 0x7f66dc1030f0 2024-06-03T01:09:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.978+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66dc069180 con 0x7f66dc1030f0 2024-06-03T01:09:44.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.980+0000 7f66d9600700 1 -- 172.21.15.94:0/1705258875 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66c8010430 con 0x7f66dc1030f0 2024-06-03T01:09:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.982+0000 7f66d9600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f66c40a2c20 0x7f66c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.982+0000 7f66d9600700 1 -- 172.21.15.94:0/1705258875 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f66c80cf7d0 con 0x7f66dc1030f0 2024-06-03T01:09:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.982+0000 7f66e0c00700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f66c40a2c20 0x7f66c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.983+0000 7f66e0c00700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f66c40a2c20 0x7f66c40a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f66d0011590 tx=0x7f66d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:44.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:44.984+0000 7f66d9600700 1 -- 172.21.15.94:0/1705258875 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f66c809c350 con 0x7f66dc1030f0 2024-06-03T01:09:45.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.254+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66dc1a7bb0 con 0x7f66dc1030f0 2024-06-03T01:09:45.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.255+0000 7f66d9600700 1 -- 172.21.15.94:0/1705258875 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f66c809bc50 con 0x7f66dc1030f0 2024-06-03T01:09:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:09:45.257 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:06:22.586009+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24478,24485,24490,24505,24524]},"id":1}]} 2024-06-03T01:09:45.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.258+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f66c40a2c20 msgr2=0x7f66c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.258+0000 7f66e3600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f66c40a2c20 0x7f66c40a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f66d0011590 tx=0x7f66d0009260 comp rx=0 tx=0).stop 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.258+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66dc1030f0 msgr2=0x7f66dc19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.258+0000 7f66e3600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66dc1030f0 0x7f66dc19b370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66c800e9f0 tx=0x7f66c800ed00 comp rx=0 tx=0).stop 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.258+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 shutdown_connections 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.258+0000 7f66e3600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f66c40a2c20 0x7f66c40a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.258+0000 7f66e3600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66dc107840 0x7f66dc19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.259+0000 7f66e3600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66dc102740 0x7f66dc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:45.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.259+0000 7f66e3600700 1 --2- 172.21.15.94:0/1705258875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66dc1030f0 0x7f66dc19b370 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:45.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.259+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 >> 172.21.15.94:0/1705258875 conn(0x7f66dc0795f0 msgr2=0x7f66dc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:45.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.259+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 shutdown_connections 2024-06-03T01:09:45.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:45.259+0000 7f66e3600700 1 -- 172.21.15.94:0/1705258875 wait complete. 2024-06-03T01:09:45.261 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-06-03T01:09:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:45 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1705258875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:09:45.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:45 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1705258875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:09:45.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:45 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1705258875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:09:46.286 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:09:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:46 smithi094 ceph-mon[32119]: pgmap v486: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:46.739 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:46 smithi107 ceph-mon[36622]: pgmap v486: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:46.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:46 smithi179 ceph-mon[37041]: pgmap v486: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:49.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 -- 172.21.15.94:0/1125442840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c8105060 msgr2=0x7fe9c8111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 --2- 172.21.15.94:0/1125442840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c8105060 0x7fe9c8111570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b0a0 tx=0x7fe9c400b3b0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 -- 172.21.15.94:0/1125442840 shutdown_connections 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 --2- 172.21.15.94:0/1125442840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c8105060 0x7fe9c8111570 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 --2- 172.21.15.94:0/1125442840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9c810a6a0 0x7fe9c810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 --2- 172.21.15.94:0/1125442840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c81046a0 0x7fe9c8104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 -- 172.21.15.94:0/1125442840 >> 172.21.15.94:0/1125442840 conn(0x7fe9c8100190 msgr2=0x7fe9c81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 -- 172.21.15.94:0/1125442840 shutdown_connections 2024-06-03T01:09:49.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.212+0000 7fe9d0800700 1 -- 172.21.15.94:0/1125442840 wait complete. 2024-06-03T01:09:49.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 Processor -- start 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 -- start start 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9c81046a0 0x7fe9c819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c8105060 0x7fe9c819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c810a6a0 0x7fe9c819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81149d0 con 0x7fe9c81046a0 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8114850 con 0x7fe9c8105060 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9d0800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8114b50 con 0x7fe9c810a6a0 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9cde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9c81046a0 0x7fe9c819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.214+0000 7fe9ce800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c810a6a0 0x7fe9c819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c810a6a0 0x7fe9c819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38688/0 (socket says 172.21.15.94:38688) 2024-06-03T01:09:49.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 -- 172.21.15.94:0/2044977005 learned_addr learned my addr 172.21.15.94:0/2044977005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:49.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9cd400700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c8105060 0x7fe9c819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:49.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 -- 172.21.15.94:0/2044977005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c8105060 msgr2=0x7fe9c819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c8105060 0x7fe9c819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 -- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9c81046a0 msgr2=0x7fe9c819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9c81046a0 0x7fe9c819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 -- 172.21.15.94:0/2044977005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c819fde0 con 0x7fe9c810a6a0 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.215+0000 7fe9ce800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c810a6a0 0x7fe9c819f6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b780 tx=0x7fe9c4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.216+0000 7fe9c2a00700 1 -- 172.21.15.94:0/2044977005 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4021070 con 0x7fe9c810a6a0 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.216+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c81a0070 con 0x7fe9c810a6a0 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.216+0000 7fe9c2a00700 1 -- 172.21.15.94:0/2044977005 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4007b80 con 0x7fe9c810a6a0 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.216+0000 7fe9c2a00700 1 -- 172.21.15.94:0/2044977005 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40052a0 con 0x7fe9c810a6a0 2024-06-03T01:09:49.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.216+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c81a7950 con 0x7fe9c810a6a0 2024-06-03T01:09:49.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.217+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c8069180 con 0x7fe9c810a6a0 2024-06-03T01:09:49.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.219+0000 7fe9c2a00700 1 -- 172.21.15.94:0/2044977005 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe9c4024070 con 0x7fe9c810a6a0 2024-06-03T01:09:49.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.220+0000 7fe9c2a00700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe9b00a2ca0 0x7fe9b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:49.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.221+0000 7fe9c2a00700 1 -- 172.21.15.94:0/2044977005 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe9c40d03e0 con 0x7fe9c810a6a0 2024-06-03T01:09:49.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.221+0000 7fe9cde00700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe9b00a2ca0 0x7fe9b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:49.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.221+0000 7fe9cde00700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe9b00a2ca0 0x7fe9b00a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009be0 tx=0x7fe9bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:49.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.223+0000 7fe9c2a00700 1 -- 172.21.15.94:0/2044977005 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe9c4007d30 con 0x7fe9c810a6a0 2024-06-03T01:09:49.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.446+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9c802dca0 con 0x7fe9c810a6a0 2024-06-03T01:09:49.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.448+0000 7fe9c2a00700 1 -- 172.21.15.94:0/2044977005 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24723 (secure 0 0 0) 0x7fe9c409cff0 con 0x7fe9c810a6a0 2024-06-03T01:09:49.450 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:09:49.450 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":100,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:06:35.341979+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:09:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe9b00a2ca0 msgr2=0x7fe9b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe9b00a2ca0 0x7fe9b00a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009be0 tx=0x7fe9bc009260 comp rx=0 tx=0).stop 2024-06-03T01:09:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c810a6a0 msgr2=0x7fe9c819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c810a6a0 0x7fe9c819f6a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400b780 tx=0x7fe9c4004d00 comp rx=0 tx=0).stop 2024-06-03T01:09:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 shutdown_connections 2024-06-03T01:09:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe9b00a2ca0 0x7fe9b00a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe9c810a6a0 0x7fe9c819f6a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9c81046a0 0x7fe9c819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 --2- 172.21.15.94:0/2044977005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c8105060 0x7fe9c819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:49.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 >> 172.21.15.94:0/2044977005 conn(0x7fe9c8100190 msgr2=0x7fe9c81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:49.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 shutdown_connections 2024-06-03T01:09:49.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:49.453+0000 7fe9d0800700 1 -- 172.21.15.94:0/2044977005 wait complete. 2024-06-03T01:09:49.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:49 smithi107 ceph-mon[36622]: pgmap v487: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:49 smithi094 ceph-mon[32119]: pgmap v487: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:49 smithi179 ceph-mon[37041]: pgmap v487: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:50.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-03T01:04:57.629349+0000', 'modified': '2024-06-03T01:06:22.586009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24505, 'mds_2': 24490, 'mds_3': 24485, 'mds_4': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14388': {'gid': 14388, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/848723917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 848723917}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 848723917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14400': {'gid': 14400, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6841/3437199777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6840', 'nonce': 3437199777}, {'type': 'v1', 'addr': '172.21.15.94:6841', 'nonce': 3437199777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6833/1330319563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 1330319563}, {'type': 'v1', 'addr': '172.21.15.107:6833', 'nonce': 1330319563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/2660538909', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 2660538909}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 2660538909}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6835/2528321857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 2528321857}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 2528321857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.179:6835/1407039746', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 1407039746}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 1407039746}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.107:6837/2537813051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 2537813051}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 2537813051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.179:6837/2233357781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 2233357781}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 2233357781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.179:6839/544081992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 544081992}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 544081992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.107:6839/1423733861', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1423733861}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1423733861}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24478, 24485, 24490, 24505, 24524]} 2024-06-03T01:09:50.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-03T01:09:50.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-03T01:09:50.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:50 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2044977005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:09:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:50 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2044977005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:09:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:50 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2044977005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:09:50.902 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:51.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:51 smithi107 ceph-mon[36622]: pgmap v488: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:51.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:51 smithi094 ceph-mon[32119]: pgmap v488: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:51.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:51 smithi179 ceph-mon[37041]: pgmap v488: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:09:53.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 -- 172.21.15.94:0/1660670196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 msgr2=0x7f2a6c06a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 --2- 172.21.15.94:0/1660670196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 0x7f2a6c06a580 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009990 tx=0x7f2a5c009ca0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 -- 172.21.15.94:0/1660670196 shutdown_connections 2024-06-03T01:09:53.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 --2- 172.21.15.94:0/1660670196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6c10ef40 0x7f2a6c111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 --2- 172.21.15.94:0/1660670196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 0x7f2a6c06a580 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 --2- 172.21.15.94:0/1660670196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a6c06aac0 0x7f2a6c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 -- 172.21.15.94:0/1660670196 >> 172.21.15.94:0/1660670196 conn(0x7f2a6c078340 msgr2=0x7f2a6c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:53.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 -- 172.21.15.94:0/1660670196 shutdown_connections 2024-06-03T01:09:53.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.217+0000 7f2a72c00700 1 -- 172.21.15.94:0/1660670196 wait complete. 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.218+0000 7f2a72c00700 1 Processor -- start 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a72c00700 1 -- start start 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a72c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 0x7f2a6c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a72c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a6c06aac0 0x7f2a6c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a72c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6c10ef40 0x7f2a6c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a72c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1149f0 con 0x7f2a6c06a1a0 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a72c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c114870 con 0x7f2a6c06aac0 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a72c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c114b70 con 0x7f2a6c10ef40 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.219+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 0x7f2a6c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a6b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a6c06aac0 0x7f2a6c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 0x7f2a6c19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34908/0 (socket says 172.21.15.94:34908) 2024-06-03T01:09:53.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a6be00700 1 -- 172.21.15.94:0/2044867778 learned_addr learned my addr 172.21.15.94:0/2044867778 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:53.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6c10ef40 0x7f2a6c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a70c00700 1 -- 172.21.15.94:0/2044867778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a6c06aac0 msgr2=0x7f2a6c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a6c06aac0 0x7f2a6c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a70c00700 1 -- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 msgr2=0x7f2a6c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 0x7f2a6c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a70c00700 1 -- 172.21.15.94:0/2044867778 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c19fe80 con 0x7f2a6c10ef40 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.220+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6c10ef40 0x7f2a6c19f740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400ef90 tx=0x7f2a6400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.221+0000 7f2a68c00700 1 -- 172.21.15.94:0/2044867778 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640092e0 con 0x7f2a6c10ef40 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.221+0000 7f2a68c00700 1 -- 172.21.15.94:0/2044867778 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a64004bb0 con 0x7f2a6c10ef40 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.221+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c1a0110 con 0x7f2a6c10ef40 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.221+0000 7f2a68c00700 1 -- 172.21.15.94:0/2044867778 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64004280 con 0x7f2a6c10ef40 2024-06-03T01:09:53.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.221+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1a79f0 con 0x7f2a6c10ef40 2024-06-03T01:09:53.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.224+0000 7f2a68c00700 1 -- 172.21.15.94:0/2044867778 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a6401b030 con 0x7f2a6c10ef40 2024-06-03T01:09:53.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.225+0000 7f2a7d54bb80 1 Processor -- start 2024-06-03T01:09:53.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.225+0000 7f2a68c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2a500a2c20 0x7f2a500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a7d54bb80 1 -- start start 2024-06-03T01:09:53.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a68c00700 1 -- 172.21.15.94:0/2044867778 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2a640cefc0 con 0x7f2a6c10ef40 2024-06-03T01:09:53.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177434b40 0x561177434f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a6b400700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2a500a2c20 0x7f2a500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177435460 0x56117742ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x56117742f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56117733d9c0 con 0x56117742f1f0 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561177380ab0 con 0x561177434b40 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56117733b8a0 con 0x561177435460 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177434b40 0x561177434f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177434b40 0x561177434f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:34146/0 (socket says 172.21.15.94:34146) 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a6be00700 1 -- 172.21.15.94:0/2890694981 learned_addr learned my addr 172.21.15.94:0/2890694981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x56117742f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.226+0000 7f2a6b400700 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177435460 0x56117742ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a6be00700 1 -- 172.21.15.94:0/2890694981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177435460 msgr2=0x56117742ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a6be00700 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177435460 0x56117742ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a6be00700 1 -- 172.21.15.94:0/2890694981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 msgr2=0x56117742f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a6be00700 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x56117742f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a6be00700 1 -- 172.21.15.94:0/2890694981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561177328f40 con 0x561177434b40 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a6b400700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2a500a2c20 0x7f2a500a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c1028f0 tx=0x7f2a580118e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a6be00700 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177434b40 0x561177434f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005c10 tx=0x7f2a5c005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd009f27f3a55c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a61600700 1 -- 172.21.15.94:0/2890694981 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00bc90 con 0x561177434b40 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a61600700 1 -- 172.21.15.94:0/2890694981 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c00be30 con 0x561177434b40 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.227+0000 7f2a61600700 1 -- 172.21.15.94:0/2890694981 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00f460 con 0x561177434b40 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2890694981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177434b40 msgr2=0x561177434f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177434b40 0x561177434f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005c10 tx=0x7f2a5c005c40 comp rx=0 tx=0).stop 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2890694981 shutdown_connections 2024-06-03T01:09:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177434b40 0x561177434f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x56117742f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2890694981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177435460 0x56117742ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2890694981 >> 172.21.15.94:0/2890694981 conn(0x56117735b120 msgr2=0x561177429fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2890694981 shutdown_connections 2024-06-03T01:09:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.228+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2890694981 wait complete. 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.229+0000 7f2a7d54bb80 1 -- start start 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.229+0000 7f2a7d54bb80 1 Processor -- start 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56117742f1f0 0x56117743b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x5611774384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177435460 0x561177438a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56117731cf10 con 0x561177435460 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56117737ca70 con 0x56117742f1f0 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56117743bde0 con 0x561177434b40 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a7d54bb80 10 client.0 _ll_register_callbacks cb 0x561177316330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56117742f1f0 0x56117743b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a70c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177435460 0x561177438a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56117742f1f0 0x56117743b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:34160/0 (socket says 172.21.15.94:34160) 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a6be00700 1 -- 172.21.15.94:0/1512509661 learned_addr learned my addr 172.21.15.94:0/1512509661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.230+0000 7f2a6b400700 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x5611774384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a6be00700 1 -- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 msgr2=0x5611774384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a6be00700 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x5611774384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a6be00700 1 -- 172.21.15.94:0/1512509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177435460 msgr2=0x561177438a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a6be00700 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177435460 0x561177438a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a6be00700 1 -- 172.21.15.94:0/1512509661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611772cf880 con 0x56117742f1f0 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a70c00700 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177435460 0x561177438a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a6be00700 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56117742f1f0 0x56117743b1e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0038f0 tx=0x7f2a5c003980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c9b34ba10f7c1d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a56000700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a56000700 1 -- 172.21.15.94:0/1512509661 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c030020 con 0x56117742f1f0 2024-06-03T01:09:53.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a56000700 1 -- 172.21.15.94:0/1512509661 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2a5c005110 con 0x56117742f1f0 2024-06-03T01:09:53.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.231+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c00be30 con 0x56117742f1f0 2024-06-03T01:09:53.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a56000700 1 -- 172.21.15.94:0/1512509661 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a5c03b050 con 0x56117742f1f0 2024-06-03T01:09:53.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 10 client.25115 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:53.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611774464f0 con 0x56117742f1f0 2024-06-03T01:09:53.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a56000700 1 -- 172.21.15.94:0/1512509661 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c00be30 con 0x56117742f1f0 2024-06-03T01:09:53.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a56000700 1 -- 172.21.15.94:0/1512509661 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2a5c02b030 con 0x56117742f1f0 2024-06-03T01:09:53.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 10 client.25115 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 10 client.25115 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 20 client.25115 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x561177448610 0x561177449a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 4 client.25115 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.232+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 --> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56117738dfb0 con 0x561177448610 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.233+0000 7f2a6b400700 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x561177448610 0x561177449a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.233+0000 7f2a56000700 10 client.25115 ms_handle_connect on v2:172.21.15.179:6834/1407039746 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.233+0000 7f2a6b400700 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x561177448610 0x561177449a80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ba37325412cee59 server_cookie=983f400c601150d3 in_seq=0 out_seq=0 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a56000700 1 -- 172.21.15.94:0/1512509661 <== mds.4 v2:172.21.15.179:6834/1407039746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56117738dfb0 con 0x561177448610 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a56000700 10 client.25115 handle_command_reply: tid=0 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 client.25115 shutdown 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56117742f1f0 msgr2=0x56117743b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56117742f1f0 0x56117743b1e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0038f0 tx=0x7f2a5c003980 comp rx=0 tx=0).stop 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 shutdown_connections 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56117742f1f0 0x56117743b1e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x561177448610 0x561177449a80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177435460 0x561177438a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/1512509661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x5611774384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a56000700 0 client.25115 ms_handle_reset on v2:172.21.15.179:6834/1407039746 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 >> 172.21.15.94:0/1512509661 conn(0x56117735b120 msgr2=0x561177423740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 shutdown_connections 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/1512509661 wait complete. 2024-06-03T01:09:53.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.235+0000 7f2a7d54bb80 20 client.25115 trim_cache size 0 max 16384 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.258+0000 7f2a7d54bb80 1 Processor -- start 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.258+0000 7f2a7d54bb80 1 -- start start 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56117742f1f0 0x5611774240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177434b40 0x5611774447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x561177444cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561177447dc0 con 0x561177434b40 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561177448490 con 0x561177435460 2024-06-03T01:09:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561177448b60 con 0x56117742f1f0 2024-06-03T01:09:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a6b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177434b40 0x5611774447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a70c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x561177444cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a6b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177434b40 0x5611774447b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34926/0 (socket says 172.21.15.94:34926) 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a6b400700 1 -- 172.21.15.94:0/2077476031 learned_addr learned my addr 172.21.15.94:0/2077476031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.259+0000 7f2a6be00700 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56117742f1f0 0x5611774240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a70c00700 1 -- 172.21.15.94:0/2077476031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56117742f1f0 msgr2=0x5611774240e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56117742f1f0 0x5611774240e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a70c00700 1 -- 172.21.15.94:0/2077476031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177434b40 msgr2=0x5611774447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177434b40 0x5611774447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a70c00700 1 -- 172.21.15.94:0/2077476031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561177435260 con 0x561177435460 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a6b400700 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177434b40 0x5611774447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a6be00700 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56117742f1f0 0x5611774240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a70c00700 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x561177444cf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a640cc6e0 tx=0x7f2a6400c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68758eda5a18580 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a60c00700 1 -- 172.21.15.94:0/2077476031 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6409ae30 con 0x561177435460 2024-06-03T01:09:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a60c00700 1 -- 172.21.15.94:0/2077476031 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6409bbd0 con 0x561177435460 2024-06-03T01:09:53.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.260+0000 7f2a60c00700 1 -- 172.21.15.94:0/2077476031 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64004d60 con 0x561177435460 2024-06-03T01:09:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2077476031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 msgr2=0x561177444cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x561177444cf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a640cc6e0 tx=0x7f2a6400c9e0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2077476031 shutdown_connections 2024-06-03T01:09:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x561177444cf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56117742f1f0 0x5611774240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/2077476031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561177434b40 0x5611774447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2077476031 >> 172.21.15.94:0/2077476031 conn(0x56117735b120 msgr2=0x5611774440a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2077476031 shutdown_connections 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.261+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/2077476031 wait complete. 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.262+0000 7f2a7d54bb80 1 -- start start 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.262+0000 7f2a7d54bb80 1 Processor -- start 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.262+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x561177453750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.262+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x56117744ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.262+0000 7f2a7d54bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x56117744f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561177449900 con 0x56117742f1f0 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561177449230 con 0x561177435460 2024-06-03T01:09:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a7d54bb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561177449fd0 con 0x561177434b40 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a7d54bb80 10 client.0 _ll_register_callbacks cb 0x561177316330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x561177453750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x561177453750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34930/0 (socket says 172.21.15.94:34930) 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 -- 172.21.15.94:0/3150308229 learned_addr learned my addr 172.21.15.94:0/3150308229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6b400700 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x56117744ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a70c00700 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x56117744f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 -- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 msgr2=0x56117744ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x56117744ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 -- 172.21.15.94:0/3150308229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 msgr2=0x56117744f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x56117744f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.263+0000 7f2a6be00700 1 -- 172.21.15.94:0/3150308229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611774bb6b0 con 0x56117742f1f0 2024-06-03T01:09:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a6be00700 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x561177453750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c001eb0 tx=0x7f2a5c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945d707eb398b7a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:53.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a56000700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a56000700 1 -- 172.21.15.94:0/3150308229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c030040 con 0x56117742f1f0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a56000700 1 -- 172.21.15.94:0/3150308229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f2a5c00f460 con 0x56117742f1f0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6409bbd0 con 0x56117742f1f0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a56000700 1 -- 172.21.15.94:0/3150308229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a5c048050 con 0x56117742f1f0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a7d54bb80 10 client.15000 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.264+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5611774bc810 con 0x56117742f1f0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a56000700 1 -- 172.21.15.94:0/3150308229 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6409bbd0 con 0x56117742f1f0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a56000700 1 -- 172.21.15.94:0/3150308229 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2a5c02b070 con 0x56117742f1f0 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a7d54bb80 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a7d54bb80 10 client.15000 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a7d54bb80 20 client.15000 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x5611774c1430 0x5611774c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a7d54bb80 4 client.15000 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 --> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] -- command(tid 0: {"prefix": "status"}) -- 0x561177388530 con 0x5611774c1430 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.265+0000 7f2a6b400700 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x5611774c1430 0x5611774c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a6b400700 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x5611774c1430 0x5611774c0140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a88353c3e9f70fc server_cookie=b21f5a11882b09c in_seq=0 out_seq=0 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a56000700 10 client.15000 ms_handle_connect on v2:172.21.15.179:6834/1407039746 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a56000700 1 -- 172.21.15.94:0/3150308229 <== mds.4 v2:172.21.15.179:6834/1407039746 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x561177388530 con 0x5611774c1430 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a56000700 10 client.15000 handle_command_reply: tid=0 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 client.15000 shutdown 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 msgr2=0x561177453750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x561177453750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c001eb0 tx=0x7f2a5c005c40 comp rx=0 tx=0).stop 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 shutdown_connections 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561177435460 0x56117744f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:6834/1407039746,v1:172.21.15.179:6835/1407039746] conn(0x5611774c1430 0x5611774c0140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56117742f1f0 0x561177453750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 --2- 172.21.15.94:0/3150308229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561177434b40 0x56117744ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.266+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 >> 172.21.15.94:0/3150308229 conn(0x56117735b120 msgr2=0x561177435ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a56000700 0 client.15000 ms_handle_reset on v2:172.21.15.179:6834/1407039746 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 shutdown_connections 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a7d54bb80 1 -- 172.21.15.94:0/3150308229 wait complete. 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a7d54bb80 20 client.15000 trim_cache size 0 max 16384 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 4, 2024-06-03T01:09:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24478, 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 216.885315854, 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 340.92365538899998 2024-06-03T01:09:53.271 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2a500a2c20 msgr2=0x7f2a500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a72c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2a500a2c20 0x7f2a500a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c1028f0 tx=0x7f2a580118e0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6c10ef40 msgr2=0x7f2a6c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.267+0000 7f2a72c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6c10ef40 0x7f2a6c19f740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400ef90 tx=0x7f2a6400c3a0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 shutdown_connections 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f2a500a2c20 0x7f2a500a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a6c10ef40 0x7f2a6c19f740 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a6c06a1a0 0x7f2a6c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 --2- 172.21.15.94:0/2044867778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a6c06aac0 0x7f2a6c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:53.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 >> 172.21.15.94:0/2044867778 conn(0x7f2a6c078340 msgr2=0x7f2a6c110390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 shutdown_connections 2024-06-03T01:09:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:53.268+0000 7f2a72c00700 1 -- 172.21.15.94:0/2044867778 wait complete. 2024-06-03T01:09:53.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:53 smithi107 ceph-mon[36622]: pgmap v489: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:53.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:53 smithi094 ceph-mon[32119]: pgmap v489: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:53.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:53 smithi179 ceph-mon[37041]: pgmap v489: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:54.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-03T01:09:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:55.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:55 smithi094 ceph-mon[32119]: pgmap v490: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:55.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:55 smithi107 ceph-mon[36622]: pgmap v490: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:55.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:55 smithi179 ceph-mon[37041]: pgmap v490: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:57.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 -- 172.21.15.94:0/2460929224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9841030f0 msgr2=0x7fb984107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 --2- 172.21.15.94:0/2460929224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9841030f0 0x7fb984107110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb970009990 tx=0x7fb970009ca0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 -- 172.21.15.94:0/2460929224 shutdown_connections 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 --2- 172.21.15.94:0/2460929224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 0x7fb984102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 --2- 172.21.15.94:0/2460929224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb984107840 0x7fb98410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 --2- 172.21.15.94:0/2460929224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9841030f0 0x7fb984107110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 -- 172.21.15.94:0/2460929224 >> 172.21.15.94:0/2460929224 conn(0x7fb9840795f0 msgr2=0x7fb984079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 -- 172.21.15.94:0/2460929224 shutdown_connections 2024-06-03T01:09:57.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.303+0000 7fb98b800700 1 -- 172.21.15.94:0/2460929224 wait complete. 2024-06-03T01:09:57.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.304+0000 7fb98b800700 1 Processor -- start 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.304+0000 7fb98b800700 1 -- start start 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb98b800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 0x7fb984074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb98b800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9841030f0 0x7fb984074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb98b800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb984107840 0x7fb984075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb98b800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb984112750 con 0x7fb9841030f0 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb98b800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841125d0 con 0x7fb984102740 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb98b800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841128d0 con 0x7fb984107840 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb983e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9841030f0 0x7fb984074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb988e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 0x7fb984074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb988e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 0x7fb984074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57114/0 (socket says 172.21.15.94:57114) 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb988e00700 1 -- 172.21.15.94:0/313199343 learned_addr learned my addr 172.21.15.94:0/313199343 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:57.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.305+0000 7fb989800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb984107840 0x7fb984075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb988e00700 1 -- 172.21.15.94:0/313199343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb984107840 msgr2=0x7fb984075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb988e00700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb984107840 0x7fb984075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb988e00700 1 -- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9841030f0 msgr2=0x7fb984074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb988e00700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9841030f0 0x7fb984074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb988e00700 1 -- 172.21.15.94:0/313199343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9841a7100 con 0x7fb984102740 2024-06-03T01:09:57.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb988e00700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 0x7fb984074790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e9a0 tx=0x7fb97800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:57.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb981600700 1 -- 172.21.15.94:0/313199343 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97800ca10 con 0x7fb984102740 2024-06-03T01:09:57.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb981600700 1 -- 172.21.15.94:0/313199343 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97800cbb0 con 0x7fb984102740 2024-06-03T01:09:57.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9841a73f0 con 0x7fb984102740 2024-06-03T01:09:57.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb981600700 1 -- 172.21.15.94:0/313199343 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978018660 con 0x7fb984102740 2024-06-03T01:09:57.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.306+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9841a7900 con 0x7fb984102740 2024-06-03T01:09:57.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.310+0000 7fb981600700 1 -- 172.21.15.94:0/313199343 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb978010430 con 0x7fb984102740 2024-06-03T01:09:57.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.311+0000 7fb996092b80 1 Processor -- start 2024-06-03T01:09:57.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.311+0000 7fb981600700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb96c0a2c20 0x7fb96c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.311+0000 7fb996092b80 1 -- start start 2024-06-03T01:09:57.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.311+0000 7fb981600700 1 -- 172.21.15.94:0/313199343 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb9780cf5c0 con 0x7fb984102740 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.311+0000 7fb989800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb96c0a2c20 0x7fb96c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb996092b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db870ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb996092b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8709cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb996092b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db870a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb996092b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562db86189c0 con 0x562db870fb40 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb996092b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562db865bab0 con 0x562db8710460 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb996092b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562db86168a0 con 0x562db870a1f0 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb988e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db870ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb983e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8709cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb988e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db870ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42016/0 (socket says 172.21.15.94:42016) 2024-06-03T01:09:57.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb983e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8709cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57126/0 (socket says 172.21.15.94:57126) 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb988e00700 1 -- 172.21.15.94:0/3447217835 learned_addr learned my addr 172.21.15.94:0/3447217835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb989800700 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db870a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb989800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb96c0a2c20 0x7fb96c0a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00f540 tx=0x7fb97c009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb989800700 1 -- 172.21.15.94:0/3447217835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 msgr2=0x562db8709cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb989800700 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8709cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb989800700 1 -- 172.21.15.94:0/3447217835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 msgr2=0x562db870ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.312+0000 7fb989800700 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db870ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.313+0000 7fb989800700 1 -- 172.21.15.94:0/3447217835 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db8603f40 con 0x562db870a1f0 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.313+0000 7fb983e00700 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8709cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:57.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.313+0000 7fb989800700 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db870a630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb97c019780 tx=0x7fb97c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46d21cb0fd6c2d68 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:57.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.313+0000 7fb976000700 1 -- 172.21.15.94:0/3447217835 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c01d070 con 0x562db870a1f0 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.313+0000 7fb976000700 1 -- 172.21.15.94:0/3447217835 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c017650 con 0x562db870a1f0 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.313+0000 7fb976000700 1 -- 172.21.15.94:0/3447217835 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c004280 con 0x562db870a1f0 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 -- 172.21.15.94:0/3447217835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 msgr2=0x562db870a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db870a630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb97c019780 tx=0x7fb97c01c000 comp rx=0 tx=0).stop 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 -- 172.21.15.94:0/3447217835 shutdown_connections 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db870a630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db870ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 --2- 172.21.15.94:0/3447217835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8709cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 -- 172.21.15.94:0/3447217835 >> 172.21.15.94:0/3447217835 conn(0x562db8636120 msgr2=0x562db8704fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 -- 172.21.15.94:0/3447217835 shutdown_connections 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.314+0000 7fb996092b80 1 -- 172.21.15.94:0/3447217835 wait complete. 2024-06-03T01:09:57.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.315+0000 7fb996092b80 1 -- start start 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.315+0000 7fb996092b80 1 Processor -- start 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb996092b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8716210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb996092b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870fb40 0x562db8713510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb996092b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8713a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb996092b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562db85f7f10 con 0x562db870a1f0 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb996092b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562db8657a70 con 0x562db8710460 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb996092b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562db8716e40 con 0x562db870fb40 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb996092b80 10 client.0 _ll_register_callbacks cb 0x562db85f1330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb988e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8716210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb988e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8716210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42032/0 (socket says 172.21.15.94:42032) 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb983e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870fb40 0x562db8713510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb988e00700 1 -- 172.21.15.94:0/3917817686 learned_addr learned my addr 172.21.15.94:0/3917817686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb989800700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8713a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb988e00700 1 -- 172.21.15.94:0/3917817686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870fb40 msgr2=0x562db8713510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb988e00700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870fb40 0x562db8713510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb988e00700 1 -- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 msgr2=0x562db8713a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.316+0000 7fb988e00700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8713a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb988e00700 1 -- 172.21.15.94:0/3917817686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db85aa880 con 0x562db870a1f0 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb983e00700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870fb40 0x562db8713510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb989800700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8713a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:09:57.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb988e00700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8716210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c2d0 tx=0x7fb97809c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bf3a8b223fce0e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:57.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb96aa00700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:09:57.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb96aa00700 1 -- 172.21.15.94:0/3917817686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780b4060 con 0x562db870a1f0 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb96aa00700 1 -- 172.21.15.94:0/3917817686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb9780a4220 con 0x562db870a1f0 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb97c017650 con 0x562db870a1f0 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb96aa00700 1 -- 172.21.15.94:0/3917817686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb9780a46b0 con 0x562db870a1f0 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb996092b80 10 client.15012 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.317+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562db87216e0 con 0x562db870a1f0 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb96aa00700 1 -- 172.21.15.94:0/3917817686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c017650 con 0x562db870a1f0 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb96aa00700 1 -- 172.21.15.94:0/3917817686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb978010720 con 0x562db870a1f0 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb996092b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb996092b80 10 client.15012 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb996092b80 20 client.15012 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb996092b80 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db8723c20 0x562db8725090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb996092b80 4 client.15012 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.318+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 --> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562db8668fb0 con 0x562db8723c20 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.319+0000 7fb983e00700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db8723c20 0x562db8725090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.319+0000 7fb983e00700 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db8723c20 0x562db8725090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6327a74b7413ac47 server_cookie=47d249cfde8fd4bf in_seq=0 out_seq=0 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.319+0000 7fb96aa00700 10 client.15012 ms_handle_connect on v2:172.21.15.107:6836/2537813051 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb96aa00700 1 -- 172.21.15.94:0/3917817686 <== mds.3 v2:172.21.15.107:6836/2537813051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562db8668fb0 con 0x562db8723c20 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb96aa00700 10 client.15012 handle_command_reply: tid=0 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 client.15012 shutdown 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 msgr2=0x562db8716210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8716210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c2d0 tx=0x7fb97809c140 comp rx=0 tx=0).stop 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 shutdown_connections 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db8713a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db8723c20 0x562db8725090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8716210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb996092b80 1 --2- 172.21.15.94:0/3917817686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870fb40 0x562db8713510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.320+0000 7fb96aa00700 0 client.15012 ms_handle_reset on v2:172.21.15.107:6836/2537813051 2024-06-03T01:09:57.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.321+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 >> 172.21.15.94:0/3917817686 conn(0x562db8636120 msgr2=0x562db86fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:57.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.321+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 shutdown_connections 2024-06-03T01:09:57.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.321+0000 7fb996092b80 1 -- 172.21.15.94:0/3917817686 wait complete. 2024-06-03T01:09:57.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.321+0000 7fb996092b80 20 client.15012 trim_cache size 0 max 16384 2024-06-03T01:09:57.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.344+0000 7fb996092b80 1 Processor -- start 2024-06-03T01:09:57.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.344+0000 7fb996092b80 1 -- start start 2024-06-03T01:09:57.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb996092b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8709cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb996092b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db870fb40 0x562db8713be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb996092b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db8710460 0x562db86ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb996092b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562db87233d0 con 0x562db870a1f0 2024-06-03T01:09:57.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb996092b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562db8723aa0 con 0x562db870fb40 2024-06-03T01:09:57.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb996092b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562db8724170 con 0x562db8710460 2024-06-03T01:09:57.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb988e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8709cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb989800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db8710460 0x562db86ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb988e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8709cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42036/0 (socket says 172.21.15.94:42036) 2024-06-03T01:09:57.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb988e00700 1 -- 172.21.15.94:0/3478431688 learned_addr learned my addr 172.21.15.94:0/3478431688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:57.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.345+0000 7fb983e00700 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db870fb40 0x562db8713be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb989800700 1 -- 172.21.15.94:0/3478431688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db870fb40 msgr2=0x562db8713be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb989800700 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db870fb40 0x562db8713be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb989800700 1 -- 172.21.15.94:0/3478431688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 msgr2=0x562db8709cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb989800700 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8709cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb989800700 1 -- 172.21.15.94:0/3478431688 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db8710260 con 0x562db8710460 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb988e00700 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8709cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb989800700 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db8710460 0x562db86ff0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb97c019750 tx=0x7fb97c004ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65ada94e41d78b16 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.346+0000 7fb975600700 1 -- 172.21.15.94:0/3478431688 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c030030 con 0x562db8710460 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb975600700 1 -- 172.21.15.94:0/3478431688 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c026040 con 0x562db8710460 2024-06-03T01:09:57.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb975600700 1 -- 172.21.15.94:0/3478431688 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c025040 con 0x562db8710460 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 -- 172.21.15.94:0/3478431688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db8710460 msgr2=0x562db86ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db8710460 0x562db86ff0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb97c019750 tx=0x7fb97c004ca0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 -- 172.21.15.94:0/3478431688 shutdown_connections 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db8710460 0x562db86ff0e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870a1f0 0x562db8709cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 --2- 172.21.15.94:0/3478431688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db870fb40 0x562db8713be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 -- 172.21.15.94:0/3478431688 >> 172.21.15.94:0/3478431688 conn(0x562db8636120 msgr2=0x562db87134d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 -- 172.21.15.94:0/3478431688 shutdown_connections 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.347+0000 7fb996092b80 1 -- 172.21.15.94:0/3478431688 wait complete. 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.348+0000 7fb996092b80 1 -- start start 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.348+0000 7fb996092b80 1 Processor -- start 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb996092b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db872e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb996092b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db872ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb996092b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db87294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb996092b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562db8724f10 con 0x562db870fb40 2024-06-03T01:09:57.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb996092b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562db8724840 con 0x562db8710460 2024-06-03T01:09:57.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb996092b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562db87255e0 con 0x562db870a1f0 2024-06-03T01:09:57.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb996092b80 10 client.0 _ll_register_callbacks cb 0x562db85f1330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb983e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db872ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db872e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db872e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43328/0 (socket says 172.21.15.94:43328) 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 -- 172.21.15.94:0/888865434 learned_addr learned my addr 172.21.15.94:0/888865434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb989800700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db87294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 -- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 msgr2=0x562db87294c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db87294c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 -- 172.21.15.94:0/888865434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 msgr2=0x562db872ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db872ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb988e00700 1 -- 172.21.15.94:0/888865434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db8729c60 con 0x562db870a1f0 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb989800700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db87294c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.349+0000 7fb983e00700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db872ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb988e00700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db872e6f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb97809b740 tx=0x7fb97809bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ddc8d483e3011 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb96aa00700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb96aa00700 1 -- 172.21.15.94:0/888865434 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780a4420 con 0x562db870a1f0 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb96aa00700 1 -- 172.21.15.94:0/888865434 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fb9780ba170 con 0x562db870a1f0 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb97c026040 con 0x562db870a1f0 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb96aa00700 1 -- 172.21.15.94:0/888865434 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb9780ba370 con 0x562db870a1f0 2024-06-03T01:09:57.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb996092b80 10 client.25141 fetch_fsmap learned FSMap version 27 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.350+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562db8798760 con 0x562db870a1f0 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb96aa00700 1 -- 172.21.15.94:0/888865434 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c026040 con 0x562db870a1f0 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb96aa00700 1 -- 172.21.15.94:0/888865434 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb97809ab10 con 0x562db870a1f0 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb996092b80 10 client.25141 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb996092b80 10 client.25141 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb996092b80 20 client.25141 populate_metadata read hostname 'smithi094' 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb996092b80 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db879d180 0x562db879be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb996092b80 4 client.25141 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:09:57.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 --> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] -- command(tid 0: {"prefix": "status"}) -- 0x562db8663530 con 0x562db879d180 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.351+0000 7fb983e00700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db879d180 0x562db879be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb983e00700 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db879d180 0x562db879be90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=817d21b2f66c6576 server_cookie=75e09b91828a2db7 in_seq=0 out_seq=0 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb96aa00700 10 client.25141 ms_handle_connect on v2:172.21.15.107:6836/2537813051 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb96aa00700 1 -- 172.21.15.94:0/888865434 <== mds.3 v2:172.21.15.107:6836/2537813051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x562db8663530 con 0x562db879d180 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb96aa00700 10 client.25141 handle_command_reply: tid=0 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 client.25141 shutdown 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 msgr2=0x562db872e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db872e6f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb97809b740 tx=0x7fb97809bbc0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 shutdown_connections 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562db8710460 0x562db87294c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.107:6836/2537813051,v1:172.21.15.107:6837/2537813051] conn(0x562db879d180 0x562db879be90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562db870a1f0 0x562db872e6f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 --2- 172.21.15.94:0/888865434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562db870fb40 0x562db872ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 >> 172.21.15.94:0/888865434 conn(0x562db8636120 msgr2=0x562db8710840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb96aa00700 0 client.25141 ms_handle_reset on v2:172.21.15.107:6836/2537813051 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.352+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 shutdown_connections 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb996092b80 1 -- 172.21.15.94:0/888865434 wait complete. 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb996092b80 20 client.25141 trim_cache size 0 max 16384 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:09:57.357 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 3, 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24485, 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 221.983897189, 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 339.26616815300002 2024-06-03T01:09:57.358 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb96c0a2c20 msgr2=0x7fb96c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb96c0a2c20 0x7fb96c0a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00f540 tx=0x7fb97c009920 comp rx=0 tx=0).stop 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 msgr2=0x7fb984074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 0x7fb984074790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e9a0 tx=0x7fb97800ecb0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 shutdown_connections 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb984102740 0x7fb984074790 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fb96c0a2c20 0x7fb96c0a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb984107840 0x7fb984075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 --2- 172.21.15.94:0/313199343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9841030f0 0x7fb984074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:09:57.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.353+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 >> 172.21.15.94:0/313199343 conn(0x7fb9840795f0 msgr2=0x7fb984100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:09:57.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.354+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 shutdown_connections 2024-06-03T01:09:57.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:09:57.354+0000 7fb98b800700 1 -- 172.21.15.94:0/313199343 wait complete. 2024-06-03T01:09:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:57 smithi094 ceph-mon[32119]: pgmap v491: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:57.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:57 smithi107 ceph-mon[36622]: pgmap v491: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:57.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:57 smithi179 ceph-mon[37041]: pgmap v491: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:09:58.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-03T01:09:58.843 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:09:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:59 smithi107 ceph-mon[36622]: pgmap v492: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:09:59 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:09:59.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:59 smithi094 ceph-mon[32119]: pgmap v492: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:59.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:09:59 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:09:59.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:59 smithi179 ceph-mon[37041]: pgmap v492: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:09:59.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:09:59 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:10:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:00 smithi107 ceph-mon[36622]: overall HEALTH_OK 2024-06-03T01:10:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:00 smithi107 ceph-mon[36622]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:10:00.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:00 smithi094 ceph-mon[32119]: overall HEALTH_OK 2024-06-03T01:10:00.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:00.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:00.815 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:00 smithi094 ceph-mon[32119]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:10:00.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:00 smithi179 ceph-mon[37041]: overall HEALTH_OK 2024-06-03T01:10:00.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:00.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:00.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:00 smithi179 ceph-mon[37041]: from='mgr.14152 172.21.15.94:0/2160764631' entity='mgr.x' 2024-06-03T01:10:01.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 -- 172.21.15.94:0/2781067125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c074830 msgr2=0x7fa16c074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 --2- 172.21.15.94:0/2781067125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c074830 0x7fa16c074c10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1600098f0 tx=0x7fa160009c00 comp rx=0 tx=0).stop 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 -- 172.21.15.94:0/2781067125 shutdown_connections 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 --2- 172.21.15.94:0/2781067125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c074830 0x7fa16c074c10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 --2- 172.21.15.94:0/2781067125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c1133b0 0x7fa16c1157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 --2- 172.21.15.94:0/2781067125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa16c075150 0x7fa16c112e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 -- 172.21.15.94:0/2781067125 >> 172.21.15.94:0/2781067125 conn(0x7fa16c0fdeb0 msgr2=0x7fa16c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 -- 172.21.15.94:0/2781067125 shutdown_connections 2024-06-03T01:10:01.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.360+0000 7fa172400700 1 -- 172.21.15.94:0/2781067125 wait complete. 2024-06-03T01:10:01.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.361+0000 7fa172400700 1 Processor -- start 2024-06-03T01:10:01.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.361+0000 7fa172400700 1 -- start start 2024-06-03T01:10:01.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa172400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c074830 0x7fa16c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa172400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa16c075150 0x7fa16c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa172400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c1133b0 0x7fa16c1a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa172400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c118cd0 con 0x7fa16c074830 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa172400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c118b50 con 0x7fa16c1133b0 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa172400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c118e50 con 0x7fa16c075150 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c074830 0x7fa16c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c074830 0x7fa16c19f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42064/0 (socket says 172.21.15.94:42064) 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171000700 1 -- 172.21.15.94:0/1642446562 learned_addr learned my addr 172.21.15.94:0/1642446562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171a00700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c1133b0 0x7fa16c1a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa16be00700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa16c075150 0x7fa16c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171000700 1 -- 172.21.15.94:0/1642446562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa16c075150 msgr2=0x7fa16c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171000700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa16c075150 0x7fa16c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171000700 1 -- 172.21.15.94:0/1642446562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c1133b0 msgr2=0x7fa16c1a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.362+0000 7fa171000700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c1133b0 0x7fa16c1a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.363+0000 7fa171000700 1 -- 172.21.15.94:0/1642446562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c1a41c0 con 0x7fa16c074830 2024-06-03T01:10:01.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.363+0000 7fa171000700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c074830 0x7fa16c19f200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa160009230 tx=0x7fa16000fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:01.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.363+0000 7fa169600700 1 -- 172.21.15.94:0/1642446562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1600051f0 con 0x7fa16c074830 2024-06-03T01:10:01.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.363+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16c1a4450 con 0x7fa16c074830 2024-06-03T01:10:01.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.363+0000 7fa169600700 1 -- 172.21.15.94:0/1642446562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16000b560 con 0x7fa16c074830 2024-06-03T01:10:01.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.363+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16c1abd30 con 0x7fa16c074830 2024-06-03T01:10:01.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.363+0000 7fa169600700 1 -- 172.21.15.94:0/1642446562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160003b30 con 0x7fa16c074830 2024-06-03T01:10:01.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.368+0000 7fa169600700 1 -- 172.21.15.94:0/1642446562 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa160029060 con 0x7fa16c074830 2024-06-03T01:10:01.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.369+0000 7fa17e426b80 1 Processor -- start 2024-06-03T01:10:01.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.369+0000 7fa169600700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1540a2c20 0x7fa1540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.369+0000 7fa17e426b80 1 -- start start 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.369+0000 7fa169600700 1 -- 172.21.15.94:0/1642446562 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa1600cfdf0 con 0x7fa16c074830 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa171a00700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1540a2c20 0x7fa1540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c2d10 0x5580732c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa17e426b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580731d29c0 con 0x5580732c2d10 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa17e426b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558073215ab0 con 0x5580732c3630 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa17e426b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5580731d08a0 con 0x5580732ce4e0 2024-06-03T01:10:01.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa171000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c2d10 0x5580732c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa171000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c2d10 0x5580732c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42066/0 (socket says 172.21.15.94:42066) 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa171000700 1 -- 172.21.15.94:0/1519223690 learned_addr learned my addr 172.21.15.94:0/1519223690 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa16be00700 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.370+0000 7fa171a00700 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa171000700 1 -- 172.21.15.94:0/1519223690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 msgr2=0x5580732d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa171000700 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa171000700 1 -- 172.21.15.94:0/1519223690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 msgr2=0x5580732cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa171000700 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa171000700 1 -- 172.21.15.94:0/1519223690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580731bdf40 con 0x5580732c2d10 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa171a00700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1540a2c20 0x7fa1540a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa164009fd0 tx=0x7fa16400fc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:01.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa171000700 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c2d10 0x5580732c30f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa1600a49b0 tx=0x7fa16000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf6950cb993628 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa15e000700 1 -- 172.21.15.94:0/1519223690 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1600a4ee0 con 0x5580732c2d10 2024-06-03T01:10:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa15e000700 1 -- 172.21.15.94:0/1519223690 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16009a600 con 0x5580732c2d10 2024-06-03T01:10:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.371+0000 7fa15e000700 1 -- 172.21.15.94:0/1519223690 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16009c560 con 0x5580732c2d10 2024-06-03T01:10:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 -- 172.21.15.94:0/1519223690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c2d10 msgr2=0x5580732c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c2d10 0x5580732c30f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa1600a49b0 tx=0x7fa16000ba10 comp rx=0 tx=0).stop 2024-06-03T01:10:01.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 -- 172.21.15.94:0/1519223690 shutdown_connections 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c2d10 0x5580732c30f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 --2- 172.21.15.94:0/1519223690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 -- 172.21.15.94:0/1519223690 >> 172.21.15.94:0/1519223690 conn(0x5580731f0120 msgr2=0x5580732bf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 -- 172.21.15.94:0/1519223690 shutdown_connections 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.372+0000 7fa17e426b80 1 -- 172.21.15.94:0/1519223690 wait complete. 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.373+0000 7fa17e426b80 1 -- start start 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.373+0000 7fa17e426b80 1 Processor -- start 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c2d10 0x5580732cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa17e426b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580731b1f10 con 0x5580732ce4e0 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa17e426b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558073211a70 con 0x5580732c3630 2024-06-03T01:10:01.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa17e426b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5580731d29c0 con 0x5580732c2d10 2024-06-03T01:10:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa17e426b80 10 client.0 _ll_register_callbacks cb 0x5580731ab330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa16be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa171a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa16be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57212/0 (socket says 172.21.15.94:57212) 2024-06-03T01:10:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa16be00700 1 -- 172.21.15.94:0/4229596356 learned_addr learned my addr 172.21.15.94:0/4229596356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:01.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.374+0000 7fa171000700 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c2d10 0x5580732cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa171a00700 1 -- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c2d10 msgr2=0x5580732cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa171a00700 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c2d10 0x5580732cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa171a00700 1 -- 172.21.15.94:0/4229596356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 msgr2=0x5580732caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa171a00700 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa171a00700 1 -- 172.21.15.94:0/4229596356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558073164880 con 0x5580732ce4e0 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa171a00700 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732cb460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa164018da0 tx=0x7fa164004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adec410c2bac3b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa152a00700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa152a00700 1 -- 172.21.15.94:0/4229596356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164008e00 con 0x5580732ce4e0 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa152a00700 1 -- 172.21.15.94:0/4229596356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa164003a00 con 0x5580732ce4e0 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.375+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa16009a600 con 0x5580732ce4e0 2024-06-03T01:10:01.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa152a00700 1 -- 172.21.15.94:0/4229596356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa164032050 con 0x5580732ce4e0 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 10 client.15030 fetch_fsmap learned FSMap version 27 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5580732db340 con 0x5580732ce4e0 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa152a00700 1 -- 172.21.15.94:0/4229596356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16009a600 con 0x5580732ce4e0 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa152a00700 1 -- 172.21.15.94:0/4229596356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa1640045a0 con 0x5580732ce4e0 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 10 client.15030 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 20 client.15030 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5580732dd9f0 0x5580732dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 4 client.15030 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.376+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 --> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558073222fb0 con 0x5580732dd9f0 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.377+0000 7fa171000700 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5580732dd9f0 0x5580732dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.377+0000 7fa171000700 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5580732dd9f0 0x5580732dee60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c611e4b769395f34 server_cookie=d8fd893d76879358 in_seq=0 out_seq=0 2024-06-03T01:10:01.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.377+0000 7fa152a00700 10 client.15030 ms_handle_connect on v2:172.21.15.179:6836/2233357781 2024-06-03T01:10:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.378+0000 7fa152a00700 1 -- 172.21.15.94:0/4229596356 <== mds.2 v2:172.21.15.179:6836/2233357781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558073222fb0 con 0x5580732dd9f0 2024-06-03T01:10:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.378+0000 7fa152a00700 10 client.15030 handle_command_reply: tid=0 2024-06-03T01:10:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.378+0000 7fa17e426b80 1 client.15030 shutdown 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 msgr2=0x5580732cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732cb460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa164018da0 tx=0x7fa164004b40 comp rx=0 tx=0).stop 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 shutdown_connections 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x5580732dd9f0 0x5580732dee60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732cb460 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c2d10 0x5580732cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 --2- 172.21.15.94:0/4229596356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c3630 0x5580732caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 >> 172.21.15.94:0/4229596356 conn(0x5580731f0120 msgr2=0x5580732bee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa152a00700 0 client.15030 ms_handle_reset on v2:172.21.15.179:6836/2233357781 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 shutdown_connections 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 1 -- 172.21.15.94:0/4229596356 wait complete. 2024-06-03T01:10:01.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.379+0000 7fa17e426b80 20 client.15030 trim_cache size 0 max 16384 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.402+0000 7fa17e426b80 1 Processor -- start 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.402+0000 7fa17e426b80 1 -- start start 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c3630 0x5580732c6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa17e426b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580732dd1a0 con 0x5580732ce4e0 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa17e426b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5580732dd870 con 0x5580732c2d10 2024-06-03T01:10:01.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa17e426b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5580732ddf40 con 0x5580732c3630 2024-06-03T01:10:01.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa16be00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c3630 0x5580732c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa16be00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c3630 0x5580732c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43368/0 (socket says 172.21.15.94:43368) 2024-06-03T01:10:01.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa16be00700 1 -- 172.21.15.94:0/2235704870 learned_addr learned my addr 172.21.15.94:0/2235704870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa171000700 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.403+0000 7fa171a00700 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa16be00700 1 -- 172.21.15.94:0/2235704870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 msgr2=0x5580732c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa16be00700 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa16be00700 1 -- 172.21.15.94:0/2235704870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 msgr2=0x5580732e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa16be00700 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732e82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa16be00700 1 -- 172.21.15.94:0/2235704870 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580732c9490 con 0x5580732c3630 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa171000700 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa16be00700 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c3630 0x5580732c6430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa158009fc0 tx=0x7fa158009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4f27224de80b47e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa15d600700 1 -- 172.21.15.94:0/2235704870 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158022070 con 0x5580732c3630 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa15d600700 1 -- 172.21.15.94:0/2235704870 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa158004530 con 0x5580732c3630 2024-06-03T01:10:01.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.404+0000 7fa15d600700 1 -- 172.21.15.94:0/2235704870 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15801d460 con 0x5580732c3630 2024-06-03T01:10:01.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 -- 172.21.15.94:0/2235704870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c3630 msgr2=0x5580732c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c3630 0x5580732c6430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa158009fc0 tx=0x7fa158009310 comp rx=0 tx=0).stop 2024-06-03T01:10:01.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 -- 172.21.15.94:0/2235704870 shutdown_connections 2024-06-03T01:10:01.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732ce4e0 0x5580732e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 --2- 172.21.15.94:0/2235704870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732c3630 0x5580732c6430 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 -- 172.21.15.94:0/2235704870 >> 172.21.15.94:0/2235704870 conn(0x5580731f0120 msgr2=0x5580732c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 -- 172.21.15.94:0/2235704870 shutdown_connections 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.405+0000 7fa17e426b80 1 -- 172.21.15.94:0/2235704870 wait complete. 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.406+0000 7fa17e426b80 1 -- start start 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.406+0000 7fa17e426b80 1 Processor -- start 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.406+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c3630 0x5580732e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa17e426b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732e3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa17e426b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5580732dece0 con 0x5580732c3630 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa17e426b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5580732de610 con 0x5580732c2d10 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa17e426b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5580732df3b0 con 0x5580732ce4e0 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa17e426b80 10 client.0 _ll_register_callbacks cb 0x5580731ab330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:01.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732e84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57240/0 (socket says 172.21.15.94:57240) 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 -- 172.21.15.94:0/157414109 learned_addr learned my addr 172.21.15.94:0/157414109 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa16be00700 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c3630 0x5580732e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171a00700 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 -- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 msgr2=0x5580732e3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732e3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 -- 172.21.15.94:0/157414109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c3630 msgr2=0x5580732e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c3630 0x5580732e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.407+0000 7fa171000700 1 -- 172.21.15.94:0/157414109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580732e3aa0 con 0x5580732c2d10 2024-06-03T01:10:01.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa171000700 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732e84c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa160005e90 tx=0x7fa160009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08cbdab6dc0064d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:01.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa152a00700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:10:01.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa152a00700 1 -- 172.21.15.94:0/157414109 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1600b5240 con 0x5580732c2d10 2024-06-03T01:10:01.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa152a00700 1 -- 172.21.15.94:0/157414109 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fa16000baa0 con 0x5580732c2d10 2024-06-03T01:10:01.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa158004530 con 0x5580732c2d10 2024-06-03T01:10:01.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa152a00700 1 -- 172.21.15.94:0/157414109 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa16009ca00 con 0x5580732c2d10 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa17e426b80 10 client.25157 fetch_fsmap learned FSMap version 27 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558073349d80 con 0x5580732c2d10 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.408+0000 7fa152a00700 1 -- 172.21.15.94:0/157414109 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa158004530 con 0x5580732c2d10 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa152a00700 1 -- 172.21.15.94:0/157414109 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa16000e470 con 0x5580732c2d10 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa17e426b80 10 client.25157 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa17e426b80 10 client.25157 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa17e426b80 20 client.25157 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa17e426b80 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x55807334e7c0 0x55807334d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa17e426b80 4 client.25157 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 --> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] -- command(tid 0: {"prefix": "status"}) -- 0x55807321d530 con 0x55807334e7c0 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.409+0000 7fa16be00700 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x55807334e7c0 0x55807334d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.410+0000 7fa152a00700 10 client.25157 ms_handle_connect on v2:172.21.15.179:6836/2233357781 2024-06-03T01:10:01.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.410+0000 7fa16be00700 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x55807334e7c0 0x55807334d4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6334d3a33593441f server_cookie=bf9081945217441d in_seq=0 out_seq=0 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.410+0000 7fa152a00700 1 -- 172.21.15.94:0/157414109 <== mds.2 v2:172.21.15.179:6836/2233357781 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55807321d530 con 0x55807334e7c0 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.410+0000 7fa152a00700 10 client.25157 handle_command_reply: tid=0 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.410+0000 7fa17e426b80 1 client.25157 shutdown 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.410+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 msgr2=0x5580732e84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.410+0000 7fa17e426b80 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732e84c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa160005e90 tx=0x7fa160009520 comp rx=0 tx=0).stop 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 shutdown_connections 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:6836/2233357781,v1:172.21.15.179:6837/2233357781] conn(0x55807334e7c0 0x55807334d4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5580732c2d10 0x5580732e84c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5580732ce4e0 0x5580732e3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 --2- 172.21.15.94:0/157414109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5580732c3630 0x5580732e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 >> 172.21.15.94:0/157414109 conn(0x5580731f0120 msgr2=0x5580732cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa152a00700 0 client.25157 ms_handle_reset on v2:172.21.15.179:6836/2233357781 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 shutdown_connections 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 1 -- 172.21.15.94:0/157414109 wait complete. 2024-06-03T01:10:01.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.411+0000 7fa17e426b80 20 client.25157 trim_cache size 0 max 16384 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24490, 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 227.072486349, 2024-06-03T01:10:01.414 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:10:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:10:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:10:01.415 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 346.13204881500002 2024-06-03T01:10:01.415 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:10:01.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1540a2c20 msgr2=0x7fa1540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1540a2c20 0x7fa1540a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa164009fd0 tx=0x7fa16400fc40 comp rx=0 tx=0).stop 2024-06-03T01:10:01.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c074830 msgr2=0x7fa16c19f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c074830 0x7fa16c19f200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa160009230 tx=0x7fa16000fb20 comp rx=0 tx=0).stop 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 shutdown_connections 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa16c1133b0 0x7fa16c1a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fa1540a2c20 0x7fa1540a50e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa16c074830 0x7fa16c19f200 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 --2- 172.21.15.94:0/1642446562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa16c075150 0x7fa16c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 >> 172.21.15.94:0/1642446562 conn(0x7fa16c0fdeb0 msgr2=0x7fa16c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 shutdown_connections 2024-06-03T01:10:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:01.412+0000 7fa172400700 1 -- 172.21.15.94:0/1642446562 wait complete. 2024-06-03T01:10:02.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-03T01:10:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:02 smithi094 ceph-mon[32119]: pgmap v493: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:10:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:02 smithi179 ceph-mon[37041]: pgmap v493: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:10:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:02 smithi107 ceph-mon[36622]: pgmap v493: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-03T01:10:02.847 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:10:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:04 smithi094 ceph-mon[32119]: pgmap v494: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:10:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:04 smithi179 ceph-mon[37041]: pgmap v494: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:10:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:04 smithi107 ceph-mon[36622]: pgmap v494: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 -- 172.21.15.94:0/1016981159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806ac60 msgr2=0x7f34e806b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 --2- 172.21.15.94:0/1016981159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806ac60 0x7f34e806b0e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34d40099c0 tx=0x7f34d4009cd0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 -- 172.21.15.94:0/1016981159 shutdown_connections 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 --2- 172.21.15.94:0/1016981159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e806b620 0x7f34e810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 --2- 172.21.15.94:0/1016981159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e81085b0 0x7f34e8108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 --2- 172.21.15.94:0/1016981159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806ac60 0x7f34e806b0e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 -- 172.21.15.94:0/1016981159 >> 172.21.15.94:0/1016981159 conn(0x7f34e80772a0 msgr2=0x7f34e80776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 -- 172.21.15.94:0/1016981159 shutdown_connections 2024-06-03T01:10:05.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.190+0000 7f34ed400700 1 -- 172.21.15.94:0/1016981159 wait complete. 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.191+0000 7f34ed400700 1 Processor -- start 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.191+0000 7f34ed400700 1 -- start start 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34ed400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e806ac60 0x7f34e8101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34ed400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806b620 0x7f34e8101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34ed400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e81085b0 0x7f34e8102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34ed400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81127f0 con 0x7f34e806b620 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34ed400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8112670 con 0x7f34e806ac60 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34ed400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8112970 con 0x7f34e81085b0 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34e5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806b620 0x7f34e8101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34e6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e806ac60 0x7f34e8101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34e5e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806b620 0x7f34e8101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41726/0 (socket says 172.21.15.94:41726) 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34e5e00700 1 -- 172.21.15.94:0/2245532059 learned_addr learned my addr 172.21.15.94:0/2245532059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.192+0000 7f34e7200700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e81085b0 0x7f34e8102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34e6800700 1 -- 172.21.15.94:0/2245532059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e81085b0 msgr2=0x7f34e8102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34e6800700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e81085b0 0x7f34e8102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34e6800700 1 -- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806b620 msgr2=0x7f34e8101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34e6800700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806b620 0x7f34e8101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34e6800700 1 -- 172.21.15.94:0/2245532059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e81ab590 con 0x7f34e806ac60 2024-06-03T01:10:05.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34e6800700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e806ac60 0x7f34e8101590 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d950 tx=0x7f34d800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:05.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34df400700 1 -- 172.21.15.94:0/2245532059 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8016070 con 0x7f34e806ac60 2024-06-03T01:10:05.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34df400700 1 -- 172.21.15.94:0/2245532059 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d800fd70 con 0x7f34e806ac60 2024-06-03T01:10:05.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34df400700 1 -- 172.21.15.94:0/2245532059 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800e3b0 con 0x7f34e806ac60 2024-06-03T01:10:05.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e81ab820 con 0x7f34e806ac60 2024-06-03T01:10:05.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.193+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e81abce0 con 0x7f34e806ac60 2024-06-03T01:10:05.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.197+0000 7f34df400700 1 -- 172.21.15.94:0/2245532059 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f34d8019070 con 0x7f34e806ac60 2024-06-03T01:10:05.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.198+0000 7f34f7d1cb80 1 Processor -- start 2024-06-03T01:10:05.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.198+0000 7f34df400700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34cc0a2ca0 0x7f34cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.198+0000 7f34f7d1cb80 1 -- start start 2024-06-03T01:10:05.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.198+0000 7f34df400700 1 -- 172.21.15.94:0/2245532059 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f34d80ced30 con 0x7f34e806ac60 2024-06-03T01:10:05.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34e7200700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34cc0a2ca0 0x7f34cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d8460 0x5613ba4d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5613ba3e09c0 con 0x5613ba4d21f0 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613ba423ab0 con 0x5613ba4d7b40 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5613ba3de8a0 con 0x5613ba4d8460 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34e6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34e6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4d7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:53408/0 (socket says 172.21.15.94:53408) 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34e6800700 1 -- 172.21.15.94:0/850193083 learned_addr learned my addr 172.21.15.94:0/850193083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34e5e00700 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d8460 0x5613ba4d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34e7200700 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.199+0000 7f34e6800700 1 -- 172.21.15.94:0/850193083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d8460 msgr2=0x5613ba4d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34e6800700 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d8460 0x5613ba4d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34e6800700 1 -- 172.21.15.94:0/850193083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 msgr2=0x5613ba4d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34e6800700 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34e6800700 1 -- 172.21.15.94:0/850193083 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613ba3cbf40 con 0x5613ba4d7b40 2024-06-03T01:10:05.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34e7200700 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:10:05.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34e7200700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34cc0a2ca0 0x7f34cc0a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003f60 tx=0x7f34e0003f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:05.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34e6800700 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4d7f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f34d8097610 tx=0x7f34d809ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17779aa6fcb682ff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:05.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34d3e00700 1 -- 172.21.15.94:0/850193083 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8020040 con 0x5613ba4d7b40 2024-06-03T01:10:05.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34d3e00700 1 -- 172.21.15.94:0/850193083 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d809b1b0 con 0x5613ba4d7b40 2024-06-03T01:10:05.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.200+0000 7f34d3e00700 1 -- 172.21.15.94:0/850193083 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80bb070 con 0x5613ba4d7b40 2024-06-03T01:10:05.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/850193083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 msgr2=0x5613ba4d7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4d7f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f34d8097610 tx=0x7f34d809ac40 comp rx=0 tx=0).stop 2024-06-03T01:10:05.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/850193083 shutdown_connections 2024-06-03T01:10:05.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4d7f20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/850193083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d8460 0x5613ba4d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/850193083 >> 172.21.15.94:0/850193083 conn(0x5613ba3fe120 msgr2=0x5613ba4ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/850193083 shutdown_connections 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.201+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/850193083 wait complete. 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.202+0000 7f34f7d1cb80 1 -- start start 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.202+0000 7f34f7d1cb80 1 Processor -- start 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.202+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.202+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d7b40 0x5613ba4db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d8460 0x5613ba4dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5613ba3bff10 con 0x5613ba4d21f0 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613ba41fa70 con 0x5613ba4d8460 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5613ba4dede0 con 0x5613ba4d7b40 2024-06-03T01:10:05.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34f7d1cb80 10 client.0 _ll_register_callbacks cb 0x5613ba3b9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4de250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41748/0 (socket says 172.21.15.94:41748) 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 -- 172.21.15.94:0/3358831617 learned_addr learned my addr 172.21.15.94:0/3358831617 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e5e00700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d7b40 0x5613ba4db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e7200700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d8460 0x5613ba4dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 -- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d7b40 msgr2=0x5613ba4db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d7b40 0x5613ba4db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 -- 172.21.15.94:0/3358831617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d8460 msgr2=0x5613ba4dba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d8460 0x5613ba4dba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e6800700 1 -- 172.21.15.94:0/3358831617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613ba372880 con 0x5613ba4d21f0 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.203+0000 7f34e7200700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d8460 0x5613ba4dba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34e6800700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4de250 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f34d809b0f0 tx=0x7f34d800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69df2f664fc69a53 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:05.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:10:05.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34d0c00700 1 -- 172.21.15.94:0/3358831617 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8099250 con 0x5613ba4d21f0 2024-06-03T01:10:05.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34d0c00700 1 -- 172.21.15.94:0/3358831617 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f34d809f120 con 0x5613ba4d21f0 2024-06-03T01:10:05.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d809b1b0 con 0x5613ba4d21f0 2024-06-03T01:10:05.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34d0c00700 1 -- 172.21.15.94:0/3358831617 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34d80b4170 con 0x5613ba4d21f0 2024-06-03T01:10:05.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34f7d1cb80 10 client.15045 fetch_fsmap learned FSMap version 27 2024-06-03T01:10:05.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.204+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613ba4e9530 con 0x5613ba4d21f0 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34d0c00700 1 -- 172.21.15.94:0/3358831617 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d809b1b0 con 0x5613ba4d21f0 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34d0c00700 1 -- 172.21.15.94:0/3358831617 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f34d809b410 con 0x5613ba4d21f0 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34f7d1cb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34f7d1cb80 10 client.15045 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34f7d1cb80 20 client.15045 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba4ea880 0x5613ba4ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34f7d1cb80 4 client.15045 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 --> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613ba430fb0 con 0x5613ba4ea880 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.205+0000 7f34e5e00700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba4ea880 0x5613ba4ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.206+0000 7f34e5e00700 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba4ea880 0x5613ba4ec9c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26151f5bd1278d4f server_cookie=8bad101def485d7d in_seq=0 out_seq=0 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.206+0000 7f34d0c00700 10 client.15045 ms_handle_connect on v2:172.21.15.179:6838/544081992 2024-06-03T01:10:05.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34d0c00700 1 -- 172.21.15.94:0/3358831617 <== mds.1 v2:172.21.15.179:6838/544081992 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5613ba430fb0 con 0x5613ba4ea880 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34d0c00700 10 client.15045 handle_command_reply: tid=0 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 client.15045 shutdown 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 msgr2=0x5613ba4de250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4de250 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f34d809b0f0 tx=0x7f34d800bf90 comp rx=0 tx=0).stop 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 shutdown_connections 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba4ea880 0x5613ba4ec9c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d8460 0x5613ba4dba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d21f0 0x5613ba4de250 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/3358831617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d7b40 0x5613ba4db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 >> 172.21.15.94:0/3358831617 conn(0x5613ba3fe120 msgr2=0x5613ba4c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.207+0000 7f34d0c00700 0 client.15045 ms_handle_reset on v2:172.21.15.179:6838/544081992 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.208+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 shutdown_connections 2024-06-03T01:10:05.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.208+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/3358831617 wait complete. 2024-06-03T01:10:05.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.208+0000 7f34f7d1cb80 20 client.15045 trim_cache size 0 max 16384 2024-06-03T01:10:05.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.231+0000 7f34f7d1cb80 1 Processor -- start 2024-06-03T01:10:05.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.231+0000 7f34f7d1cb80 1 -- start start 2024-06-03T01:10:05.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4d8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5613ba4ead00 con 0x5613ba4d8460 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613ba4eb3d0 con 0x5613ba4d7b40 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5613ba4ebaa0 con 0x5613ba4d21f0 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:46242/0 (socket says 172.21.15.94:46242) 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 -- 172.21.15.94:0/4070197664 learned_addr learned my addr 172.21.15.94:0/4070197664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e7200700 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4d8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e5e00700 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 -- 172.21.15.94:0/4070197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 msgr2=0x5613ba4c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 -- 172.21.15.94:0/4070197664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 msgr2=0x5613ba4d8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4d8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e6800700 1 -- 172.21.15.94:0/4070197664 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613ba4ce730 con 0x5613ba4d21f0 2024-06-03T01:10:05.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.232+0000 7f34e5e00700 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:10:05.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.233+0000 7f34e6800700 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4d1cb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000c00 tx=0x7f34d8010870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65d846971ccfad6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.233+0000 7f34d3400700 1 -- 172.21.15.94:0/4070197664 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8020070 con 0x5613ba4d21f0 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.233+0000 7f34d3400700 1 -- 172.21.15.94:0/4070197664 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d809b1b0 con 0x5613ba4d21f0 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.233+0000 7f34d3400700 1 -- 172.21.15.94:0/4070197664 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d809b410 con 0x5613ba4d21f0 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/4070197664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 msgr2=0x5613ba4d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4d1cb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f34d8000c00 tx=0x7f34d8010870 comp rx=0 tx=0).stop 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/4070197664 shutdown_connections 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4d8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4d1cb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/4070197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/4070197664 >> 172.21.15.94:0/4070197664 conn(0x5613ba3fe120 msgr2=0x5613ba4d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/4070197664 shutdown_connections 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/4070197664 wait complete. 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 -- start start 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.234+0000 7f34f7d1cb80 1 Processor -- start 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4dcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34f7d1cb80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5613ba4ec840 con 0x5613ba4d8460 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5613ba4ec170 con 0x5613ba4d7b40 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34f7d1cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5613ba4ecf10 con 0x5613ba4d21f0 2024-06-03T01:10:05.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34f7d1cb80 10 client.0 _ll_register_callbacks cb 0x5613ba3b9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34e7200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34e6800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4dcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34e6800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4dcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:46258/0 (socket says 172.21.15.94:46258) 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34e7200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4dd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41760/0 (socket says 172.21.15.94:41760) 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34e6800700 1 -- 172.21.15.94:0/2721428924 learned_addr learned my addr 172.21.15.94:0/2721428924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.235+0000 7f34e5e00700 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34e6800700 1 -- 172.21.15.94:0/2721428924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 msgr2=0x5613ba4dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34e6800700 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34e6800700 1 -- 172.21.15.94:0/2721428924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 msgr2=0x5613ba4dd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34e6800700 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4dd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34e6800700 1 -- 172.21.15.94:0/2721428924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613ba4f4eb0 con 0x5613ba4d21f0 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34e6800700 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4dcd80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f34d80a38f0 tx=0x7f34d80a3d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d3123d075ae0d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:05.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34d0c00700 1 -- 172.21.15.94:0/2721428924 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8020070 con 0x5613ba4d21f0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34d0c00700 1 -- 172.21.15.94:0/2721428924 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f34d809f120 con 0x5613ba4d21f0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.236+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d809b1b0 con 0x5613ba4d21f0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34d0c00700 1 -- 172.21.15.94:0/2721428924 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34d809a930 con 0x5613ba4d21f0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 10 client.25177 fetch_fsmap learned FSMap version 27 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613ba4f4eb0 con 0x5613ba4d21f0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34d0c00700 1 -- 172.21.15.94:0/2721428924 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d809b1b0 con 0x5613ba4d21f0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34d0c00700 1 -- 172.21.15.94:0/2721428924 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f34d8096f00 con 0x5613ba4d21f0 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 10 client.25177 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:10:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 10 client.25177 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 20 client.25177 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba55a510 0x5613ba559220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 4 client.25177 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 --> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] -- command(tid 0: {"prefix": "status"}) -- 0x5613ba42b530 con 0x5613ba55a510 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.237+0000 7f34e5e00700 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba55a510 0x5613ba559220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34e5e00700 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba55a510 0x5613ba559220 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fb6ee3246c61ae0 server_cookie=ae05e88453b3c37f in_seq=0 out_seq=0 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34d0c00700 10 client.25177 ms_handle_connect on v2:172.21.15.179:6838/544081992 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34d0c00700 1 -- 172.21.15.94:0/2721428924 <== mds.1 v2:172.21.15.179:6838/544081992 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5613ba42b530 con 0x5613ba55a510 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34d0c00700 10 client.25177 handle_command_reply: tid=0 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 client.25177 shutdown 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 msgr2=0x5613ba4dcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4dcd80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f34d80a38f0 tx=0x7f34d80a3d10 comp rx=0 tx=0).stop 2024-06-03T01:10:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 shutdown_connections 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:6838/544081992,v1:172.21.15.179:6839/544081992] conn(0x5613ba55a510 0x5613ba559220 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613ba4d8460 0x5613ba4dd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ba4d21f0 0x5613ba4dcd80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 --2- 172.21.15.94:0/2721428924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5613ba4d7b40 0x5613ba4dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34d0c00700 0 client.25177 ms_handle_reset on v2:172.21.15.179:6838/544081992 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.238+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 >> 172.21.15.94:0/2721428924 conn(0x5613ba3fe120 msgr2=0x5613ba4d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.239+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 shutdown_connections 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.239+0000 7f34f7d1cb80 1 -- 172.21.15.94:0/2721428924 wait complete. 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.239+0000 7f34f7d1cb80 20 client.25177 trim_cache size 0 max 16384 2024-06-03T01:10:05.243 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24505, 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 231.91219897600001, 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 344.21917705300001 2024-06-03T01:10:05.244 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34cc0a2ca0 msgr2=0x7f34cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34cc0a2ca0 0x7f34cc0a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003f60 tx=0x7f34e0003f90 comp rx=0 tx=0).stop 2024-06-03T01:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e806ac60 msgr2=0x7f34e8101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e806ac60 0x7f34e8101590 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d950 tx=0x7f34d800dc60 comp rx=0 tx=0).stop 2024-06-03T01:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 shutdown_connections 2024-06-03T01:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e806ac60 0x7f34e8101590 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f34cc0a2ca0 0x7f34cc0a5160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34e81085b0 0x7f34e8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 --2- 172.21.15.94:0/2245532059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f34e806b620 0x7f34e8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:05.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 >> 172.21.15.94:0/2245532059 conn(0x7f34e80772a0 msgr2=0x7f34e80fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:05.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 shutdown_connections 2024-06-03T01:10:05.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:05.240+0000 7f34ed400700 1 -- 172.21.15.94:0/2245532059 wait complete. 2024-06-03T01:10:06.153 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-03T01:10:06.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:06 smithi094 ceph-mon[32119]: pgmap v495: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:10:06.598 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:10:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:06 smithi179 ceph-mon[37041]: pgmap v495: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:10:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:06 smithi107 ceph-mon[36622]: pgmap v495: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:10:08.447 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:08 smithi094 ceph-mon[32119]: pgmap v496: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:10:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:08 smithi179 ceph-mon[37041]: pgmap v496: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:10:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:08 smithi107 ceph-mon[36622]: pgmap v496: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-03T01:10:08.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 -- 172.21.15.94:0/594797728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c102740 msgr2=0x7f7a4c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:08.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 --2- 172.21.15.94:0/594797728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c102740 0x7f7a4c102b20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7a400099c0 tx=0x7f7a40009cd0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 -- 172.21.15.94:0/594797728 shutdown_connections 2024-06-03T01:10:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 --2- 172.21.15.94:0/594797728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a4c107840 0x7f7a4c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 --2- 172.21.15.94:0/594797728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c102740 0x7f7a4c102b20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 --2- 172.21.15.94:0/594797728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a4c1030f0 0x7f7a4c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 -- 172.21.15.94:0/594797728 >> 172.21.15.94:0/594797728 conn(0x7f7a4c0795f0 msgr2=0x7f7a4c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 -- 172.21.15.94:0/594797728 shutdown_connections 2024-06-03T01:10:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.977+0000 7f7a53e00700 1 -- 172.21.15.94:0/594797728 wait complete. 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.978+0000 7f7a53e00700 1 Processor -- start 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.978+0000 7f7a53e00700 1 -- start start 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a53e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a4c102740 0x7f7a4c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a53e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c1030f0 0x7f7a4c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a53e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a4c107840 0x7f7a4c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a53e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c112780 con 0x7f7a4c1030f0 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a53e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c112600 con 0x7f7a4c107840 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a53e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c112900 con 0x7f7a4c102740 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a50a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c1030f0 0x7f7a4c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a50a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c1030f0 0x7f7a4c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41768/0 (socket says 172.21.15.94:41768) 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a50a00700 1 -- 172.21.15.94:0/275494510 learned_addr learned my addr 172.21.15.94:0/275494510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a51400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a4c102740 0x7f7a4c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.979+0000 7f7a51e00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a4c107840 0x7f7a4c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a50a00700 1 -- 172.21.15.94:0/275494510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a4c102740 msgr2=0x7f7a4c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a50a00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a4c102740 0x7f7a4c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a50a00700 1 -- 172.21.15.94:0/275494510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a4c107840 msgr2=0x7f7a4c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a50a00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a4c107840 0x7f7a4c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a50a00700 1 -- 172.21.15.94:0/275494510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c199140 con 0x7f7a4c1030f0 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a51400700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a4c102740 0x7f7a4c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a50a00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c1030f0 0x7f7a4c19ed60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800d950 tx=0x7f7a3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:08.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a4a000700 1 -- 172.21.15.94:0/275494510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38016070 con 0x7f7a4c1030f0 2024-06-03T01:10:08.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a4a000700 1 -- 172.21.15.94:0/275494510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3800fd70 con 0x7f7a4c1030f0 2024-06-03T01:10:08.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c199430 con 0x7f7a4c1030f0 2024-06-03T01:10:08.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a4a000700 1 -- 172.21.15.94:0/275494510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800e610 con 0x7f7a4c1030f0 2024-06-03T01:10:08.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.980+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c1a56f0 con 0x7f7a4c1030f0 2024-06-03T01:10:08.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.984+0000 7f7a4a000700 1 -- 172.21.15.94:0/275494510 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a38019070 con 0x7f7a4c1030f0 2024-06-03T01:10:08.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.985+0000 7f7a4a000700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7a340a2c20 0x7f7a340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.985+0000 7f7a5e691b80 1 Processor -- start 2024-06-03T01:10:08.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.985+0000 7f7a4a000700 1 -- 172.21.15.94:0/275494510 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7a380cee70 con 0x7f7a4c1030f0 2024-06-03T01:10:08.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.985+0000 7f7a5e691b80 1 -- start start 2024-06-03T01:10:08.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.985+0000 7f7a51400700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7a340a2c20 0x7f7a340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1ce560 0x560cef1ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1dd6c0 0x560cef1dfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef0dd9c0 con 0x560cef1dd6c0 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560cef120ab0 con 0x560cef1ce560 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560cef0db8a0 con 0x560cef14adf0 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a51e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1dd6c0 0x560cef1dfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a51e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1dd6c0 0x560cef1dfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41782/0 (socket says 172.21.15.94:41782) 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a51e00700 1 -- 172.21.15.94:0/3560399062 learned_addr learned my addr 172.21.15.94:0/3560399062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:08.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a50a00700 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1ce560 0x560cef1ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a51400700 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a51400700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7a340a2c20 0x7f7a340a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000c00 tx=0x7f7a40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a50a00700 1 -- 172.21.15.94:0/3560399062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 msgr2=0x560cef1ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a50a00700 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a50a00700 1 -- 172.21.15.94:0/3560399062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1dd6c0 msgr2=0x560cef1dfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a50a00700 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1dd6c0 0x560cef1dfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.986+0000 7f7a50a00700 1 -- 172.21.15.94:0/3560399062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cef0c8f40 con 0x560cef1ce560 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.987+0000 7f7a50a00700 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1ce560 0x560cef1ce9e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800dfb0 tx=0x7f7a38010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f0901ddd4c6c8f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.987+0000 7f7a3ea00700 1 -- 172.21.15.94:0/3560399062 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38020050 con 0x560cef1ce560 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.987+0000 7f7a3ea00700 1 -- 172.21.15.94:0/3560399062 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3809f700 con 0x560cef1ce560 2024-06-03T01:10:08.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.987+0000 7f7a3ea00700 1 -- 172.21.15.94:0/3560399062 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380a3b30 con 0x560cef1ce560 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 -- 172.21.15.94:0/3560399062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1ce560 msgr2=0x560cef1ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1ce560 0x560cef1ce9e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800dfb0 tx=0x7f7a38010b10 comp rx=0 tx=0).stop 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 -- 172.21.15.94:0/3560399062 shutdown_connections 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1dd6c0 0x560cef1dfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/3560399062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1ce560 0x560cef1ce9e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 -- 172.21.15.94:0/3560399062 >> 172.21.15.94:0/3560399062 conn(0x560cef0fb120 msgr2=0x560cef1ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 -- 172.21.15.94:0/3560399062 shutdown_connections 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.988+0000 7f7a5e691b80 1 -- 172.21.15.94:0/3560399062 wait complete. 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 -- start start 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 Processor -- start 2024-06-03T01:10:08.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef14adf0 0x560cef1dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1ce560 0x560cef1da0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef0bcf10 con 0x560cef14adf0 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560cef11ca70 con 0x560cef1dd6c0 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560cef1e0320 con 0x560cef1ce560 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.989+0000 7f7a5e691b80 10 client.0 _ll_register_callbacks cb 0x560cef0ba6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef14adf0 0x560cef1dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1da630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:53498/0 (socket says 172.21.15.94:53498) 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 -- 172.21.15.94:0/731350266 learned_addr learned my addr 172.21.15.94:0/731350266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:08.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a50a00700 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1ce560 0x560cef1da0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 -- 172.21.15.94:0/731350266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1ce560 msgr2=0x560cef1da0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1ce560 0x560cef1da0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 -- 172.21.15.94:0/731350266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef14adf0 msgr2=0x560cef1dcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef14adf0 0x560cef1dcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 -- 172.21.15.94:0/731350266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cef06f880 con 0x560cef1dd6c0 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a51e00700 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1da630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a440046c0 tx=0x7f7a440049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1787c41646968bcb server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.990+0000 7f7a33400700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.991+0000 7f7a33400700 1 -- 172.21.15.94:0/731350266 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a440148b0 con 0x560cef1dd6c0 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.991+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3809f700 con 0x560cef1dd6c0 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.991+0000 7f7a33400700 1 -- 172.21.15.94:0/731350266 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7a44014a50 con 0x560cef1dd6c0 2024-06-03T01:10:08.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.991+0000 7f7a33400700 1 -- 172.21.15.94:0/731350266 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a44027050 con 0x560cef1dd6c0 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.991+0000 7f7a5e691b80 10 client.25184 fetch_fsmap learned FSMap version 27 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.991+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560cef1eaae0 con 0x560cef1dd6c0 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.991+0000 7f7a33400700 1 -- 172.21.15.94:0/731350266 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3809f700 con 0x560cef1dd6c0 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a33400700 1 -- 172.21.15.94:0/731350266 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a4401e420 con 0x560cef1dd6c0 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a5e691b80 10 client.25184 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a5e691b80 10 client.25184 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a5e691b80 20 client.25184 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef1ed110 0x560cef1ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a5e691b80 4 client.25184 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cef12dfb0 con 0x560cef1ed110 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.992+0000 7f7a51400700 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef1ed110 0x560cef1ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.993+0000 7f7a51400700 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef1ed110 0x560cef1ee580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c8fb2efaa35836 server_cookie=9dc050aa82e0fe24 in_seq=0 out_seq=0 2024-06-03T01:10:08.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.993+0000 7f7a33400700 10 client.25184 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:08.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a33400700 1 -- 172.21.15.94:0/731350266 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560cef12dfb0 con 0x560cef1ed110 2024-06-03T01:10:08.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a33400700 10 client.25184 handle_command_reply: tid=0 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 client.25184 shutdown 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 msgr2=0x560cef1da630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1da630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a440046c0 tx=0x7f7a440049d0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 shutdown_connections 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef1ed110 0x560cef1ee580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1da630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef14adf0 0x560cef1dcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/731350266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1ce560 0x560cef1da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a33400700 0 client.25184 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 >> 172.21.15.94:0/731350266 conn(0x560cef0fb120 msgr2=0x560cef1c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 shutdown_connections 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 1 -- 172.21.15.94:0/731350266 wait complete. 2024-06-03T01:10:08.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:08.994+0000 7f7a5e691b80 20 client.25184 trim_cache size 0 max 16384 2024-06-03T01:10:09.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.017+0000 7f7a5e691b80 1 Processor -- start 2024-06-03T01:10:09.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.017+0000 7f7a5e691b80 1 -- start start 2024-06-03T01:10:09.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.017+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef14adf0 0x560cef1deab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:09.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.017+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:09.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.017+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1dd6c0 0x560cef1d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:09.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef1ec8c0 con 0x560cef1ce560 2024-06-03T01:10:09.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560cef1ecf90 con 0x560cef14adf0 2024-06-03T01:10:09.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560cef1ed660 con 0x560cef1dd6c0 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a50a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1dd6c0 0x560cef1d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a50a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41810/0 (socket says 172.21.15.94:41810) 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a50a00700 1 -- 172.21.15.94:0/1571830548 learned_addr learned my addr 172.21.15.94:0/1571830548 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51400700 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef14adf0 0x560cef1deab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51e00700 1 -- 172.21.15.94:0/1571830548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef14adf0 msgr2=0x560cef1deab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51e00700 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef14adf0 0x560cef1deab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51e00700 1 -- 172.21.15.94:0/1571830548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 msgr2=0x560cef1d89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51e00700 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51e00700 1 -- 172.21.15.94:0/1571830548 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cef1e0320 con 0x560cef1dd6c0 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a50a00700 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.018+0000 7f7a51400700 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef14adf0 0x560cef1deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.019+0000 7f7a51e00700 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1dd6c0 0x560cef1d8f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400b010 tx=0x7f7a44009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bdbd6b0eca3a98 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.019+0000 7f7a3e000700 1 -- 172.21.15.94:0/1571830548 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4402f020 con 0x560cef1dd6c0 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.019+0000 7f7a3e000700 1 -- 172.21.15.94:0/1571830548 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a440092e0 con 0x560cef1dd6c0 2024-06-03T01:10:09.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.019+0000 7f7a3e000700 1 -- 172.21.15.94:0/1571830548 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44014b10 con 0x560cef1dd6c0 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1571830548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1dd6c0 msgr2=0x560cef1d8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1dd6c0 0x560cef1d8f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400b010 tx=0x7f7a44009b20 comp rx=0 tx=0).stop 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1571830548 shutdown_connections 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef14adf0 0x560cef1deab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef1dd6c0 0x560cef1d8f20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1571830548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1571830548 >> 172.21.15.94:0/1571830548 conn(0x560cef0fb120 msgr2=0x560cef1c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1571830548 shutdown_connections 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1571830548 wait complete. 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.020+0000 7f7a5e691b80 1 -- start start 2024-06-03T01:10:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 1 Processor -- start 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef1ee400 con 0x560cef1ce560 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560cef1edd30 con 0x560cef1dd6c0 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560cef1eead0 con 0x560cef14adf0 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a5e691b80 10 client.0 _ll_register_callbacks cb 0x560cef0ba6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a51400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a51400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:46306/0 (socket says 172.21.15.94:46306) 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a51400700 1 -- 172.21.15.94:0/1936880620 learned_addr learned my addr 172.21.15.94:0/1936880620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a50a00700 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.021+0000 7f7a51e00700 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a51400700 1 -- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 msgr2=0x560cef1d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:09.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a51400700 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a51400700 1 -- 172.21.15.94:0/1936880620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 msgr2=0x560cef1d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a51400700 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a51400700 1 -- 172.21.15.94:0/1936880620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cef1d0db0 con 0x560cef14adf0 2024-06-03T01:10:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a51400700 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1d4f30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a400099c0 tx=0x7f7a40025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2ad7f690e944ba server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:09.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a33400700 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a33400700 1 -- 172.21.15.94:0/1936880620 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4002b040 con 0x560cef14adf0 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a440092e0 con 0x560cef14adf0 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.022+0000 7f7a33400700 1 -- 172.21.15.94:0/1936880620 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f7a40003860 con 0x560cef14adf0 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a33400700 1 -- 172.21.15.94:0/1936880620 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a40025e50 con 0x560cef14adf0 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 10 client.25195 fetch_fsmap learned FSMap version 27 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560cef25d190 con 0x560cef14adf0 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a33400700 1 -- 172.21.15.94:0/1936880620 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a440092e0 con 0x560cef14adf0 2024-06-03T01:10:09.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a33400700 1 -- 172.21.15.94:0/1936880620 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7a4000f460 con 0x560cef14adf0 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 10 client.25195 fetch_fsmap finished waiting for FSMap version 27 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 10 client.25195 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 20 client.25195 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef261c80 0x560cef260990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 4 client.25195 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.023+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "status"}) -- 0x560cef128530 con 0x560cef261c80 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a50a00700 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef261c80 0x560cef260990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a50a00700 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef261c80 0x560cef260990 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c4f3700783052f server_cookie=92aecafc5868447e in_seq=0 out_seq=0 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a33400700 10 client.25195 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a33400700 1 -- 172.21.15.94:0/1936880620 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x560cef128530 con 0x560cef261c80 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a33400700 10 client.25195 handle_command_reply: tid=0 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a5e691b80 1 client.25195 shutdown 2024-06-03T01:10:09.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 msgr2=0x560cef1d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.024+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1d4f30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a400099c0 tx=0x7f7a40025000 comp rx=0 tx=0).stop 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 shutdown_connections 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x560cef261c80 0x560cef260990 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560cef1dd6c0 0x560cef1d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560cef14adf0 0x560cef1d4f30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 --2- 172.21.15.94:0/1936880620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef1ce560 0x560cef1d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 >> 172.21.15.94:0/1936880620 conn(0x560cef0fb120 msgr2=0x560cef1db920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a33400700 0 client.25195 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 shutdown_connections 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 1 -- 172.21.15.94:0/1936880620 wait complete. 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a5e691b80 20 client.25195 trim_cache size 0 max 16384 2024-06-03T01:10:09.028 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "d037ccd0-2143-11ef-bc9b-c7b262605968", 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24524, 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 311.34136325200001, 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 27, 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 100, 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 98, 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 335.427500544 2024-06-03T01:10:09.029 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:10:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7a340a2c20 msgr2=0x7f7a340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a53e00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7a340a2c20 0x7f7a340a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000c00 tx=0x7f7a40005fb0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.025+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c1030f0 msgr2=0x7f7a4c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c1030f0 0x7f7a4c19ed60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800d950 tx=0x7f7a3800dc60 comp rx=0 tx=0).stop 2024-06-03T01:10:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 shutdown_connections 2024-06-03T01:10:09.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a4c107840 0x7f7a4c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f7a340a2c20 0x7f7a340a50e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a4c102740 0x7f7a4c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 --2- 172.21.15.94:0/275494510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a4c1030f0 0x7f7a4c19ed60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 >> 172.21.15.94:0/275494510 conn(0x7f7a4c0795f0 msgr2=0x7f7a4c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 shutdown_connections 2024-06-03T01:10:09.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:09.026+0000 7f7a53e00700 1 -- 172.21.15.94:0/275494510 wait complete. 2024-06-03T01:10:10.081 INFO:teuthology.run_tasks:Running task sequential... 2024-06-03T01:10:10.093 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-03T01:10:10.093 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-03T01:10:10.094 INFO:teuthology.task.print:Enabling mgr modules 2024-06-03T01:10:10.094 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-06-03T01:10:10.095 INFO:teuthology.task.exec:Executing custom commands... 2024-06-03T01:10:10.095 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:10:10.095 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-06-03T01:10:10.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.301+0000 7fe68a200640 1 -- 172.21.15.94:0/177797727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684107e50 msgr2=0x7fe684108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:10.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.301+0000 7fe68a200640 1 --2- 172.21.15.94:0/177797727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684107e50 0x7fe684108270 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe674009900 tx=0x7fe67402a3f0 comp rx=0 tx=0).stop 2024-06-03T01:10:10.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.302+0000 7fe68a200640 1 -- 172.21.15.94:0/177797727 shutdown_connections 2024-06-03T01:10:10.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.302+0000 7fe68a200640 1 --2- 172.21.15.94:0/177797727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe684109960 0x7fe684110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:10.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.302+0000 7fe68a200640 1 --2- 172.21.15.94:0/177797727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe684108fa0 0x7fe684109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:10.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.302+0000 7fe68a200640 1 --2- 172.21.15.94:0/177797727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684107e50 0x7fe684108270 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:10.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.302+0000 7fe68a200640 1 -- 172.21.15.94:0/177797727 >> 172.21.15.94:0/177797727 conn(0x7fe684076e30 msgr2=0x7fe684079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:10.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.302+0000 7fe68a200640 1 -- 172.21.15.94:0/177797727 shutdown_connections 2024-06-03T01:10:10.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.302+0000 7fe68a200640 1 -- 172.21.15.94:0/177797727 wait complete. 2024-06-03T01:10:10.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 Processor -- start 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 -- start start 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe684107e50 0x7fe68419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe684108fa0 0x7fe68419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684109960 0x7fe6841a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe684113af0 con 0x7fe684109960 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe684113970 con 0x7fe684108fa0 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe68a200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe684113c70 con 0x7fe684107e50 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe689800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684109960 0x7fe6841a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe689800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684109960 0x7fe6841a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41826/0 (socket says 172.21.15.94:41826) 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe689800640 1 -- 172.21.15.94:0/2232898172 learned_addr learned my addr 172.21.15.94:0/2232898172 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.303+0000 7fe688e00640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe684107e50 0x7fe68419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:10.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe67be00640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe684108fa0 0x7fe68419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe689800640 1 -- 172.21.15.94:0/2232898172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe684107e50 msgr2=0x7fe68419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe689800640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe684107e50 0x7fe68419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe689800640 1 -- 172.21.15.94:0/2232898172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe684108fa0 msgr2=0x7fe68419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe689800640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe684108fa0 0x7fe68419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe689800640 1 -- 172.21.15.94:0/2232898172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6841a5d80 con 0x7fe684109960 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe689800640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684109960 0x7fe6841a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00efd0 tx=0x7fe67c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe682000640 1 -- 172.21.15.94:0/2232898172 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c003b70 con 0x7fe684109960 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe682000640 1 -- 172.21.15.94:0/2232898172 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c003d10 con 0x7fe684109960 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.304+0000 7fe682000640 1 -- 172.21.15.94:0/2232898172 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c0109f0 con 0x7fe684109960 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.305+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6841a6070 con 0x7fe684109960 2024-06-03T01:10:10.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.305+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6841a6570 con 0x7fe684109960 2024-06-03T01:10:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.306+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68410f160 con 0x7fe684109960 2024-06-03T01:10:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.308+0000 7fe682000640 1 -- 172.21.15.94:0/2232898172 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe67c004020 con 0x7fe684109960 2024-06-03T01:10:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.309+0000 7fe682000640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe6680a2c00 0x7fe6680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.309+0000 7fe682000640 1 -- 172.21.15.94:0/2232898172 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fe67c0cf7e0 con 0x7fe684109960 2024-06-03T01:10:10.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.309+0000 7fe688e00640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe6680a2c00 0x7fe6680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:10.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.310+0000 7fe688e00640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe6680a2c00 0x7fe6680a50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe6740314e0 tx=0x7fe674000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:10.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.313+0000 7fe682000640 1 -- 172.21.15.94:0/2232898172 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe67c09c360 con 0x7fe684109960 2024-06-03T01:10:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:10.567+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe6840625a0 con 0x7fe684109960 2024-06-03T01:10:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:10 smithi094 ceph-mon[32119]: pgmap v497: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:10:10.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:10 smithi179 ceph-mon[37041]: pgmap v497: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:10:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:10 smithi107 ceph-mon[36622]: pgmap v497: 129 pgs: 129 active+clean; 652 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-06-03T01:10:11.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.350+0000 7fe682000640 1 -- 172.21.15.94:0/2232898172 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fe67c09bc60 con 0x7fe684109960 2024-06-03T01:10:11.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe6680a2c00 msgr2=0x7fe6680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:11.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe6680a2c00 0x7fe6680a50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe6740314e0 tx=0x7fe674000bc0 comp rx=0 tx=0).stop 2024-06-03T01:10:11.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684109960 msgr2=0x7fe6841a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:11.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684109960 0x7fe6841a5710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00efd0 tx=0x7fe67c00c6a0 comp rx=0 tx=0).stop 2024-06-03T01:10:11.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 shutdown_connections 2024-06-03T01:10:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fe6680a2c00 0x7fe6680a50c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684109960 0x7fe6841a5710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe684108fa0 0x7fe68419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 --2- 172.21.15.94:0/2232898172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe684107e50 0x7fe68419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.354+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 >> 172.21.15.94:0/2232898172 conn(0x7fe684076e30 msgr2=0x7fe68410c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.355+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 shutdown_connections 2024-06-03T01:10:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:11.355+0000 7fe68a200640 1 -- 172.21.15.94:0/2232898172 wait complete. 2024-06-03T01:10:11.388 INFO:teuthology.run_tasks:Running task kclient... 2024-06-03T01:10:11.401 INFO:tasks.kclient:Mounting kernel clients... 2024-06-03T01:10:11.401 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-03T01:10:11.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:11.401 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-06-03T01:10:11.442 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-03T01:10:11.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:11.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-03T01:10:11.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:11.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-06-03T01:10:11.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:11 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2232898172' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-03T01:10:11.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:11 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: ignoring --setuser ceph since I am not root 2024-06-03T01:10:11.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:11 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: ignoring --setgroup ceph since I am not root 2024-06-03T01:10:11.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:11 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2232898172' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-03T01:10:11.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: ignoring --setuser ceph since I am not root 2024-06-03T01:10:11.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: ignoring --setgroup ceph since I am not root 2024-06-03T01:10:11.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:11 smithi179 ceph-mgr[38906]: -- 172.21.15.179:0/3063287301 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c2db405a0 con 0x557c2db14c00 2024-06-03T01:10:11.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:11.537+0000 7fe9b9459200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T01:10:11.665 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-06-03T01:10:11.665 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-03T01:10:11.665 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:11.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-03T01:10:11.715 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:11 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2232898172' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-03T01:10:11.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: ignoring --setuser ceph since I am not root 2024-06-03T01:10:11.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: ignoring --setgroup ceph since I am not root 2024-06-03T01:10:11.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:11 smithi107 ceph-mgr[38020]: -- 172.21.15.107:0/585575280 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6489bc5a0 con 0x55a648990800 2024-06-03T01:10:11.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:11.532+0000 7fe7ff543200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T01:10:11.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:11.715+0000 7fe7ff543200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T01:10:11.949 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:11 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:11.693+0000 7f052e92e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-03T01:10:12.032 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:11.818+0000 7fe7ff543200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T01:10:12.085 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:10:12.105 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:11.719+0000 7fe9b9459200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T01:10:12.105 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:11.821+0000 7fe9b9459200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T01:10:12.231 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:11 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:11.948+0000 7f052e92e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-03T01:10:12.232 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:12 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:12.104+0000 7f052e92e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-03T01:10:12.527 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:12 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2232898172' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-03T01:10:12.527 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:12 smithi094 ceph-mon[32119]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T01:10:12.532 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:12 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2232898172' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-03T01:10:12.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:12 smithi107 ceph-mon[36622]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T01:10:12.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:12.128+0000 7fe7ff543200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T01:10:12.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:12.213+0000 7fe7ff543200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T01:10:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:12 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2232898172' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-03T01:10:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:12 smithi179 ceph-mon[37041]: mgrmap e19: x(active, since 13m), standbys: z, y 2024-06-03T01:10:12.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:12.128+0000 7fe9b9459200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T01:10:12.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:12.213+0000 7fe9b9459200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T01:10:12.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:12 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:12.525+0000 7f052e92e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-03T01:10:12.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:12 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:12.644+0000 7f052e92e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-03T01:10:13.726 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:13.390+0000 7fe7ff543200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T01:10:13.726 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:13.559+0000 7fe7ff543200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T01:10:13.734 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:13.393+0000 7fe9b9459200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T01:10:13.734 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:13.563+0000 7fe9b9459200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T01:10:13.734 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:13.733+0000 7fe9b9459200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T01:10:14.283 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:13.726+0000 7fe7ff543200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T01:10:14.283 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:14.013+0000 7fe7ff543200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T01:10:14.283 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:14.105+0000 7fe7ff543200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T01:10:14.318 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:14.022+0000 7fe9b9459200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T01:10:14.318 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:14.115+0000 7fe9b9459200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T01:10:14.338 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:14 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:14.221+0000 7f052e92e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-03T01:10:14.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.500+0000 7f1babe00700 1 -- 172.21.15.94:0/2145750205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba41030f0 msgr2=0x7f1ba4107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.500+0000 7f1babe00700 1 --2- 172.21.15.94:0/2145750205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba41030f0 0x7f1ba4107110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009920 tx=0x7f1b90009c30 comp rx=0 tx=0).stop 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.501+0000 7f1babe00700 1 -- 172.21.15.94:0/2145750205 shutdown_connections 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.501+0000 7f1babe00700 1 --2- 172.21.15.94:0/2145750205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4102740 0x7f1ba4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.501+0000 7f1babe00700 1 --2- 172.21.15.94:0/2145750205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba4107840 0x7f1ba410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.501+0000 7f1babe00700 1 --2- 172.21.15.94:0/2145750205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba41030f0 0x7f1ba4107110 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.501+0000 7f1babe00700 1 -- 172.21.15.94:0/2145750205 >> 172.21.15.94:0/2145750205 conn(0x7f1ba40795f0 msgr2=0x7f1ba4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.501+0000 7f1babe00700 1 -- 172.21.15.94:0/2145750205 shutdown_connections 2024-06-03T01:10:14.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.501+0000 7f1babe00700 1 -- 172.21.15.94:0/2145750205 wait complete. 2024-06-03T01:10:14.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 Processor -- start 2024-06-03T01:10:14.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 -- start start 2024-06-03T01:10:14.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba4102740 0x7f1ba4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:14.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba41030f0 0x7f1ba4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4107840 0x7f1ba4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba41127a0 con 0x7f1ba4102740 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4112620 con 0x7f1ba4107840 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.502+0000 7f1babe00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4112920 con 0x7f1ba41030f0 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba8a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba41030f0 0x7f1ba4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4107840 0x7f1ba4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba4102740 0x7f1ba4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba8a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba41030f0 0x7f1ba4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:46342/0 (socket says 172.21.15.94:46342) 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba8a00700 1 -- 172.21.15.94:0/1524383422 learned_addr learned my addr 172.21.15.94:0/1524383422 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4107840 0x7f1ba4075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:53570/0 (socket says 172.21.15.94:53570) 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9e00700 1 -- 172.21.15.94:0/1524383422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba41030f0 msgr2=0x7f1ba4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9e00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba41030f0 0x7f1ba4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9e00700 1 -- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba4102740 msgr2=0x7f1ba4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:14.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9e00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba4102740 0x7f1ba4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba9e00700 1 -- 172.21.15.94:0/1524383422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41a7180 con 0x7f1ba4107840 2024-06-03T01:10:14.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.503+0000 7f1ba8a00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba41030f0 0x7f1ba4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:14.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.504+0000 7f1ba9e00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4107840 0x7f1ba4075210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00efc0 tx=0x7f1b9c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:14.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.504+0000 7f1ba2000700 1 -- 172.21.15.94:0/1524383422 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0092e0 con 0x7f1ba4107840 2024-06-03T01:10:14.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.504+0000 7f1ba2000700 1 -- 172.21.15.94:0/1524383422 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b9c004bb0 con 0x7f1ba4107840 2024-06-03T01:10:14.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.504+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41a7410 con 0x7f1ba4107840 2024-06-03T01:10:14.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.504+0000 7f1ba2000700 1 -- 172.21.15.94:0/1524383422 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c004280 con 0x7f1ba4107840 2024-06-03T01:10:14.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.504+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41a7950 con 0x7f1ba4107840 2024-06-03T01:10:14.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.505+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4069180 con 0x7f1ba4107840 2024-06-03T01:10:14.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.507+0000 7f1ba2000700 1 -- 172.21.15.94:0/1524383422 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f1b9c01b030 con 0x7f1ba4107840 2024-06-03T01:10:14.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.508+0000 7f1ba2000700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 0x7f1b8c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:14.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.508+0000 7f1ba9400700 1 -- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 msgr2=0x7f1b8c0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/4120992823 2024-06-03T01:10:14.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.508+0000 7f1ba2000700 1 -- 172.21.15.94:0/1524383422 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f1b9c0ceec0 con 0x7f1ba4107840 2024-06-03T01:10:14.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.508+0000 7f1ba9400700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 0x7f1b8c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:10:14.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.512+0000 7f1ba2000700 1 -- 172.21.15.94:0/1524383422 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1b9c022020 con 0x7f1ba4107840 2024-06-03T01:10:14.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:14 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:14.463+0000 7f052e92e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-03T01:10:14.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:14.318+0000 7fe9b9459200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T01:10:14.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:14.517+0000 7fe9b9459200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T01:10:14.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.709+0000 7f1ba9400700 1 -- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 msgr2=0x7f1b8c0a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/4120992823 2024-06-03T01:10:14.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.709+0000 7f1ba9400700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 0x7f1b8c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:10:14.782 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:14.305+0000 7fe7ff543200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T01:10:14.783 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:14.501+0000 7fe7ff543200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T01:10:14.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.787+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ba402dd40 con 0x7f1ba4107840 2024-06-03T01:10:14.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.788+0000 7f1ba2000700 1 -- 172.21.15.94:0/1524383422 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f1b9c010040 con 0x7f1ba4107840 2024-06-03T01:10:14.788 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 msgr2=0x7f1b8c0a51b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 0x7f1b8c0a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4107840 msgr2=0x7f1ba4075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4107840 0x7f1ba4075210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00efc0 tx=0x7f1b9c00c3a0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 shutdown_connections 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ba4107840 0x7f1ba4075210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f1b8c0a2cf0 0x7f1b8c0a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ba4102740 0x7f1ba4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 --2- 172.21.15.94:0/1524383422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba41030f0 0x7f1ba4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.791+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 >> 172.21.15.94:0/1524383422 conn(0x7f1ba40795f0 msgr2=0x7f1ba41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.792+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 shutdown_connections 2024-06-03T01:10:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:14.792+0000 7f1babe00700 1 -- 172.21.15.94:0/1524383422 wait complete. 2024-06-03T01:10:15.077 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:14 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:14.687+0000 7f052e92e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-03T01:10:15.078 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:14 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1524383422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:10:15.104 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:14 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1524383422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:10:15.104 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:14 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1524383422' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-03T01:10:15.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:15 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:15.076+0000 7f052e92e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-03T01:10:15.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:15 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:15.205+0000 7f052e92e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-03T01:10:15.530 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:15.104+0000 7fe7ff543200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T01:10:15.537 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:15.117+0000 7fe9b9459200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T01:10:15.728 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:15 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:15.465+0000 7f052e92e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-03T01:10:15.735 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-03T01:10:15.736 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-03T01:10:15.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:15.736 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-06-03T01:10:15.783 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:15.530+0000 7fe7ff543200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T01:10:15.783 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:15.627+0000 7fe7ff543200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T01:10:15.785 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-03T01:10:15.785 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-03T01:10:15.785 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-06-03T01:10:15.785 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:10:15.785 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-06-03T01:10:15.785 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:10:15.785 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-03T01:10:15.786 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 41740sec preferred_lft 41740sec 2024-06-03T01:10:15.787 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-06-03T01:10:15.787 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-06-03T01:10:15.788 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-03T01:10:15.788 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:10:15.788 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:10:15.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-06-03T01:10:15.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-06-03T01:10:15.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-06-03T01:10:15.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-03T01:10:15.788 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:10:15.835 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:15.537+0000 7fe9b9459200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T01:10:15.835 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:15.634+0000 7fe9b9459200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T01:10:15.835 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:15.835+0000 7fe9b9459200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T01:10:15.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:10:15.987 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:15 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:15.727+0000 7f052e92e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-03T01:10:16.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:16.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:16.067 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-03T01:10:16.126 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-06-03T01:10:16.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:16.130 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-06-03T01:10:16.175 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-06-03T01:10:16.175 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-03T01:10:16.175 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-03T01:10:16.175 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-03T01:10:16.176 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-03T01:10:16.176 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:10:16.176 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:10:16.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-03T01:10:16.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-03T01:10:16.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-03T01:10:16.176 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:10:16.282 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:15.829+0000 7fe7ff543200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T01:10:16.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:10:16.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:16.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:16.414 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-06-03T01:10:16.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:16.457 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-06-03T01:10:16.548 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:10:16.548 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:10:16.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-03T01:10:16.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-03T01:10:16.549 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:10:16.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:16 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:16.561+0000 7f052e92e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-03T01:10:16.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:10:16.686 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:16 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:16.351+0000 7fe7ff543200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T01:10:16.686 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:16 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:16.443+0000 7fe7ff543200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T01:10:16.693 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:16.357+0000 7fe9b9459200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T01:10:16.694 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:16.450+0000 7fe9b9459200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T01:10:16.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:16.715 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-03T01:10:16.715 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:10:16.715 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:10:16.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-03T01:10:16.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-03T01:10:16.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-03T01:10:16.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-03T01:10:16.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-03T01:10:16.716 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:10:16.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:10:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:16.962 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-03T01:10:16.962 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-06-03T01:10:16.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-06-03T01:10:16.962 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-06-03T01:10:16.962 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-06-03T01:10:17.029 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:16 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:16.686+0000 7fe7ff543200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T01:10:17.029 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:16 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:16.818+0000 7fe7ff543200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T01:10:17.029 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:16 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:16.914+0000 7fe7ff543200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T01:10:17.039 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:16.694+0000 7fe9b9459200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T01:10:17.040 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:16.827+0000 7fe9b9459200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T01:10:17.040 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:16.923+0000 7fe9b9459200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T01:10:17.040 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:17.040+0000 7fe9b9459200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T01:10:17.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:10:17.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:17.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:10:17.145 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:17.184 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-03T01:10:17.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-03T01:10:17.185 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:17.268 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-03T01:10:17.268 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-03T01:10:17.268 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/57911953-237f-46f0-b619-4974b41e3cd6 2024-06-03T01:10:17.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:17.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/57911953-237f-46f0-b619-4974b41e3cd6 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-03T01:10:17.282 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:17 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:17.028+0000 7fe7ff543200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T01:10:17.289 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:17 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:17.155+0000 7f052e92e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-03T01:10:17.289 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:17 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:17.288+0000 7f052e92e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-03T01:10:17.394 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-03T01:10:17.394 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-06-03T01:10:17.395 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-06-03T01:10:17.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:17.341+0000 7fe9b9459200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T01:10:17.783 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:17 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:17.327+0000 7fe7ff543200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T01:10:17.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:17 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:17.567+0000 7f052e92e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-03T01:10:18.207 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-03T01:10:18.207 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-06-03T01:10:18.208 INFO:teuthology.orchestra.run.smithi094.stdout:invalid new device string format 2024-06-03T01:10:18.208 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.107:3300,172.21.15.179:3300" 2024-06-03T01:10:18.208 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.94:3300,172.21.15.107:3300,172.21.15.179:3300:/volumes/qa/sv_1/57911953-237f-46f0-b619-4974b41e3cd6 2024-06-03T01:10:18.208 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=d037ccd0-2143-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-03T01:10:18.210 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-03T01:10:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-06-03T01:10:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-03T01:10:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-06-03T01:10:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-03T01:10:18.216 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-03T01:10:18.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:10:18.216 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:18.224 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:17.949+0000 7fe9b9459200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T01:10:18.224 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z[38902]: 2024-06-03T01:10:18.167+0000 7fe9b9459200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T01:10:18.225 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:17 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:17.933+0000 7fe7ff543200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T01:10:18.225 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y[38016]: 2024-06-03T01:10:18.151+0000 7fe7ff543200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T01:10:18.282 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:10:18.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:10:18.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:18.388 INFO:teuthology.orchestra.run.smithi094.stdout:client25222 2024-06-03T01:10:18.389 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T01:10:18.389 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:10:18.435 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: Standby manager daemon y restarted 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: Standby manager daemon y started 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: Standby manager daemon z restarted 2024-06-03T01:10:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: Standby manager daemon z started 2024-06-03T01:10:18.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T01:10:18.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:10:18.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T01:10:18.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:18 smithi107 ceph-mon[36622]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:10:18.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:10:18.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:18.591 INFO:teuthology.orchestra.run.smithi094.stdout:client25222 2024-06-03T01:10:18.592 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-03T01:10:18.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:18.254+0000 7f052e92e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-03T01:10:18.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:18.384+0000 7f052e92e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-03T01:10:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: Standby manager daemon y restarted 2024-06-03T01:10:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: Standby manager daemon y started 2024-06-03T01:10:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T01:10:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:10:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T01:10:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:10:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: Standby manager daemon z restarted 2024-06-03T01:10:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: Standby manager daemon z started 2024-06-03T01:10:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T01:10:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:10:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T01:10:18.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-mon[32119]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: Standby manager daemon y restarted 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: Standby manager daemon y started 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.107:0/3297186892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: Standby manager daemon z restarted 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: Standby manager daemon z started 2024-06-03T01:10:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-03T01:10:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-03T01:10:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-03T01:10:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:18 smithi179 ceph-mon[37041]: from='mgr.? 172.21.15.179:0/1008424439' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-03T01:10:18.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-03T01:10:18.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:18.698 INFO:teuthology.orchestra.run.smithi094.stdout:client25222 2024-06-03T01:10:18.698 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968.client25222/status') 2024-06-03T01:10:18.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-03T01:10:18.864 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.25222 (3)172.21.15.94:0/1971292502 2024-06-03T01:10:18.865 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-06-03T01:10:18.865 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-06-03T01:10:18.865 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-06-03T01:10:18.865 INFO:teuthology.orchestra.run.smithi094.stderr:71 bytes copied, 7.1999e-05 s, 986 kB/s 2024-06-03T01:10:18.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-03T01:10:18.877 INFO:teuthology.run_tasks:Running task exec... 2024-06-03T01:10:18.887 INFO:teuthology.task.exec:Executing custom commands... 2024-06-03T01:10:18.887 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:10:18.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-03T01:10:19.057 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:18.725+0000 7f052e92e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-03T01:10:19.058 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:18 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:18.920+0000 7f052e92e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.082+0000 7fc69f200640 1 -- 172.21.15.94:0/3934165455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 msgr2=0x7fc6a0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.082+0000 7fc69f200640 1 --2- 172.21.15.94:0/3934165455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 0x7fc6a0109c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc698009ca0 tx=0x7fc69802a8c0 comp rx=0 tx=0).stop 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.082+0000 7fc69f200640 1 -- 172.21.15.94:0/3934165455 shutdown_connections 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.082+0000 7fc69f200640 1 --2- 172.21.15.94:0/3934165455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 0x7fc6a0109c70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.082+0000 7fc69f200640 1 --2- 172.21.15.94:0/3934165455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a0102a20 0x7fc6a0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.082+0000 7fc69f200640 1 --2- 172.21.15.94:0/3934165455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 0x7fc6a0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.082+0000 7fc69f200640 1 -- 172.21.15.94:0/3934165455 >> 172.21.15.94:0/3934165455 conn(0x7fc6a00fcda0 msgr2=0x7fc6a00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.083+0000 7fc69f200640 1 -- 172.21.15.94:0/3934165455 shutdown_connections 2024-06-03T01:10:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.083+0000 7fc69f200640 1 -- 172.21.15.94:0/3934165455 wait complete. 2024-06-03T01:10:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.083+0000 7fc69f200640 1 Processor -- start 2024-06-03T01:10:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69f200640 1 -- start start 2024-06-03T01:10:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 0x7fc6a0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a0102a20 0x7fc6a019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 0x7fc6a01a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:19.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a010d4b0 con 0x7fc6a01033e0 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a010d330 con 0x7fc6a0101830 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a010d630 con 0x7fc6a0102a20 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 0x7fc6a0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 0x7fc6a0199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41638/0 (socket says 172.21.15.94:41638) 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69de00640 1 -- 172.21.15.94:0/2419313487 learned_addr learned my addr 172.21.15.94:0/2419313487 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69d400640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a0102a20 0x7fc6a019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.084+0000 7fc69e800640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 0x7fc6a01a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69de00640 1 -- 172.21.15.94:0/2419313487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a0102a20 msgr2=0x7fc6a019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69de00640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a0102a20 0x7fc6a019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69de00640 1 -- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 msgr2=0x7fc6a01a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69de00640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 0x7fc6a01a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69de00640 1 -- 172.21.15.94:0/2419313487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a01a1930 con 0x7fc6a0101830 2024-06-03T01:10:19.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69de00640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 0x7fc6a0199d20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc69000e9c0 tx=0x7fc69000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc696a00640 1 -- 172.21.15.94:0/2419313487 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69000cd80 con 0x7fc6a0101830 2024-06-03T01:10:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a01a1c20 con 0x7fc6a0101830 2024-06-03T01:10:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc696a00640 1 -- 172.21.15.94:0/2419313487 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6900035f0 con 0x7fc6a0101830 2024-06-03T01:10:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc696a00640 1 -- 172.21.15.94:0/2419313487 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690010690 con 0x7fc6a0101830 2024-06-03T01:10:19.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.085+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a01a21c0 con 0x7fc6a0101830 2024-06-03T01:10:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.086+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a01a2390 con 0x7fc6a0101830 2024-06-03T01:10:19.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.088+0000 7fc696a00640 1 -- 172.21.15.94:0/2419313487 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7fc6900037a0 con 0x7fc6a0101830 2024-06-03T01:10:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.089+0000 7fc696a00640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 0x7fc6840a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.090+0000 7fc69d400640 1 -- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 msgr2=0x7fc6840a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/4120992823 2024-06-03T01:10:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.090+0000 7fc696a00640 1 -- 172.21.15.94:0/2419313487 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7fc6900cf8e0 con 0x7fc6a0101830 2024-06-03T01:10:19.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.090+0000 7fc69d400640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 0x7fc6840a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:10:19.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.094+0000 7fc696a00640 1 -- 172.21.15.94:0/2419313487 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc69009c460 con 0x7fc6a0101830 2024-06-03T01:10:19.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.291+0000 7fc69d400640 1 -- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 msgr2=0x7fc6840a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/4120992823 2024-06-03T01:10:19.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.291+0000 7fc69d400640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 0x7fc6840a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:10:19.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.336+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fc6a01a2780 con 0x7fc6a0101830 2024-06-03T01:10:19.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:19 smithi094 ceph-mon[32119]: mgrmap e20: x(active, since 13m), standbys: y, z 2024-06-03T01:10:19.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:19 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:19.057+0000 7f052e92e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-03T01:10:19.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:19 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:19.219+0000 7f052e92e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-03T01:10:19.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:19 smithi107 ceph-mon[36622]: mgrmap e20: x(active, since 13m), standbys: y, z 2024-06-03T01:10:19.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:19 smithi179 ceph-mon[37041]: mgrmap e20: x(active, since 13m), standbys: y, z 2024-06-03T01:10:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.692+0000 7fc69d400640 1 -- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 msgr2=0x7fc6840a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/4120992823 2024-06-03T01:10:19.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:19.692+0000 7fc69d400640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 0x7fc6840a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-03T01:10:19.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:19 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:19.594+0000 7f052e92e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-03T01:10:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.241+0000 7fc696a00640 1 -- 172.21.15.94:0/2419313487 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fc69009bd60 con 0x7fc6a0101830 2024-06-03T01:10:20.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 msgr2=0x7fc6840a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:10:20.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 0x7fc6840a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 msgr2=0x7fc6a0199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 0x7fc6a0199d20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc69000e9c0 tx=0x7fc69000ee80 comp rx=0 tx=0).stop 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 shutdown_connections 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7fc6840a2bd0 0x7fc6840a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6a01033e0 0x7fc6a01a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6a0102a20 0x7fc6a019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 --2- 172.21.15.94:0/2419313487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a0101830 0x7fc6a0199d20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 >> 172.21.15.94:0/2419313487 conn(0x7fc6a00fcda0 msgr2=0x7fc6a0073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:20.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 shutdown_connections 2024-06-03T01:10:20.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.244+0000 7fc69f200640 1 -- 172.21.15.94:0/2419313487 wait complete. 2024-06-03T01:10:20.264 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-03T01:10:20.275 INFO:teuthology.run_tasks:Running task exec... 2024-06-03T01:10:20.302 INFO:teuthology.task.exec:Executing custom commands... 2024-06-03T01:10:20.302 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:10:20.302 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-06-03T01:10:20.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 -- 172.21.15.94:0/1394485214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c401024a0 msgr2=0x7f6c40104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 --2- 172.21.15.94:0/1394485214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c401024a0 0x7f6c40104890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c34009900 tx=0x7f6c3402a410 comp rx=0 tx=0).stop 2024-06-03T01:10:20.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 -- 172.21.15.94:0/1394485214 shutdown_connections 2024-06-03T01:10:20.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 --2- 172.21.15.94:0/1394485214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c40107700 0x7f6c40109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 --2- 172.21.15.94:0/1394485214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c40104dd0 0x7f6c401071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 --2- 172.21.15.94:0/1394485214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c401024a0 0x7f6c40104890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 -- 172.21.15.94:0/1394485214 >> 172.21.15.94:0/1394485214 conn(0x7f6c400fbe90 msgr2=0x7f6c400fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:20.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 -- 172.21.15.94:0/1394485214 shutdown_connections 2024-06-03T01:10:20.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.495+0000 7f6c48800640 1 -- 172.21.15.94:0/1394485214 wait complete. 2024-06-03T01:10:20.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.496+0000 7f6c48800640 1 Processor -- start 2024-06-03T01:10:20.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c48800640 1 -- start start 2024-06-03T01:10:20.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c48800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c401024a0 0x7f6c40199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:20.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c48800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c40104dd0 0x7f6c4019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:20.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c48800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c40107700 0x7f6c401a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:20.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c48800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4010d6f0 con 0x7f6c40104dd0 2024-06-03T01:10:20.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c48800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4010d570 con 0x7f6c40107700 2024-06-03T01:10:20.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c48800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4010d870 con 0x7f6c401024a0 2024-06-03T01:10:20.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c45400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c40104dd0 0x7f6c4019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:20.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c45400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c40104dd0 0x7f6c4019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48158/0 (socket says 172.21.15.94:48158) 2024-06-03T01:10:20.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c45400640 1 -- 172.21.15.94:0/3527570999 learned_addr learned my addr 172.21.15.94:0/3527570999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:20.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c45e00640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c401024a0 0x7f6c40199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:20.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.497+0000 7f6c46800640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c40107700 0x7f6c401a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:20.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c45400640 1 -- 172.21.15.94:0/3527570999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c401024a0 msgr2=0x7f6c40199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c45400640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c401024a0 0x7f6c40199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c45400640 1 -- 172.21.15.94:0/3527570999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c40107700 msgr2=0x7f6c401a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c45400640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c40107700 0x7f6c401a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c45400640 1 -- 172.21.15.94:0/3527570999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c401a1a80 con 0x7f6c40104dd0 2024-06-03T01:10:20.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c45400640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c40104dd0 0x7f6c4019a3a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e9c0 tx=0x7f6c3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:20.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000cd80 con 0x7f6c40104dd0 2024-06-03T01:10:20.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c401a1d70 con 0x7f6c40104dd0 2024-06-03T01:10:20.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c30005500 con 0x7f6c40104dd0 2024-06-03T01:10:20.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30010690 con 0x7f6c40104dd0 2024-06-03T01:10:20.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.498+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c401a2260 con 0x7f6c40104dd0 2024-06-03T01:10:20.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.500+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c40109250 con 0x7f6c40104dd0 2024-06-03T01:10:20.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.502+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f6c300046c0 con 0x7f6c40104dd0 2024-06-03T01:10:20.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.503+0000 7f6c3aa00640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 0x7f6c280a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:20.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.504+0000 7f6c45e00640 1 -- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 msgr2=0x7f6c280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/4120992823 2024-06-03T01:10:20.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.504+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9129+0+0 (secure 0 0 0) 0x7f6c300cf800 con 0x7f6c40104dd0 2024-06-03T01:10:20.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.504+0000 7f6c45e00640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 0x7f6c280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-03T01:10:20.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.508+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6c3009c380 con 0x7f6c40104dd0 2024-06-03T01:10:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:20 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2419313487' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-03T01:10:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:20 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-03T01:10:20.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:20 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:20.370+0000 7f052e92e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-03T01:10:20.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:20 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2419313487' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-03T01:10:20.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:20 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-03T01:10:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:20 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2419313487' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-03T01:10:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:20 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-03T01:10:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.705+0000 7f6c45e00640 1 -- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 msgr2=0x7f6c280a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/4120992823 2024-06-03T01:10:20.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.705+0000 7f6c45e00640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 0x7f6c280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-03T01:10:20.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.706+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f6c300982f0 con 0x7f6c40104dd0 2024-06-03T01:10:20.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.706+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 msgr2=0x7f6c280a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-03T01:10:20.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.706+0000 7f6c3aa00640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 0x7f6c280a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.727+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f6c4002d2e0 con 0x7f6c40104dd0 2024-06-03T01:10:20.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.731+0000 7f6c3aa00640 1 -- 172.21.15.94:0/3527570999 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v33)=0 v33) ==== 129+0+0 (secure 0 0 0) 0x7f6c30057290 con 0x7f6c40104dd0 2024-06-03T01:10:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c40104dd0 msgr2=0x7f6c4019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c40104dd0 0x7f6c4019a3a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e9c0 tx=0x7f6c3000ee80 comp rx=0 tx=0).stop 2024-06-03T01:10:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 shutdown_connections 2024-06-03T01:10:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c40107700 0x7f6c401a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:6800/4120992823,v1:172.21.15.94:6801/4120992823] conn(0x7f6c280a2c80 0x7f6c280a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c40104dd0 0x7f6c4019a3a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 --2- 172.21.15.94:0/3527570999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c401024a0 0x7f6c40199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 >> 172.21.15.94:0/3527570999 conn(0x7f6c400fbe90 msgr2=0x7f6c40101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:20.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 shutdown_connections 2024-06-03T01:10:20.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.734+0000 7f6c48800640 1 -- 172.21.15.94:0/3527570999 wait complete. 2024-06-03T01:10:20.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-06-03T01:10:20.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.948+0000 7f0a67e00640 1 -- 172.21.15.94:0/4068952325 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540319b0 con 0x7f0a68103a60 2024-06-03T01:10:20.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 -- 172.21.15.94:0/4068952325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 msgr2=0x7f0a68103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 --2- 172.21.15.94:0/4068952325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 0x7f0a68103e80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009900 tx=0x7f0a5402a410 comp rx=0 tx=0).stop 2024-06-03T01:10:20.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 -- 172.21.15.94:0/4068952325 shutdown_connections 2024-06-03T01:10:20.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 --2- 172.21.15.94:0/4068952325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68105610 0x7f0a6810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 --2- 172.21.15.94:0/4068952325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a68104c50 0x7f0a681050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 --2- 172.21.15.94:0/4068952325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 0x7f0a68103e80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 -- 172.21.15.94:0/4068952325 >> 172.21.15.94:0/4068952325 conn(0x7f0a680fefd0 msgr2=0x7f0a68101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:20.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 -- 172.21.15.94:0/4068952325 shutdown_connections 2024-06-03T01:10:20.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.949+0000 7f0a6f000640 1 -- 172.21.15.94:0/4068952325 wait complete. 2024-06-03T01:10:20.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.950+0000 7f0a6f000640 1 Processor -- start 2024-06-03T01:10:20.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.950+0000 7f0a6f000640 1 -- start start 2024-06-03T01:10:20.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 0x7f0a68199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:20.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6f000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68104c50 0x7f0a6819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:20.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6f000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a68105610 0x7f0a681a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:20.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6f000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810f6d0 con 0x7f0a68104c50 2024-06-03T01:10:20.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6f000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810f550 con 0x7f0a68105610 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6810f850 con 0x7f0a68103a60 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68104c50 0x7f0a6819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6dc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 0x7f0a68199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68104c50 0x7f0a6819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48256/0 (socket says 172.21.15.94:48256) 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6d200640 1 -- 172.21.15.94:0/512426748 learned_addr learned my addr 172.21.15.94:0/512426748 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6e600640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a68105610 0x7f0a681a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6d200640 1 -- 172.21.15.94:0/512426748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 msgr2=0x7f0a68199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6d200640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 0x7f0a68199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.951+0000 7f0a6d200640 1 -- 172.21.15.94:0/512426748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a68105610 msgr2=0x7f0a681a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.952+0000 7f0a6d200640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a68105610 0x7f0a681a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.952+0000 7f0a6d200640 1 -- 172.21.15.94:0/512426748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6806b4f0 con 0x7f0a68104c50 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.952+0000 7f0a6d200640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68104c50 0x7f0a6819a2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800d770 tx=0x7f0a5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:20.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.953+0000 7f0a66000640 1 -- 172.21.15.94:0/512426748 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58017070 con 0x7f0a68104c50 2024-06-03T01:10:20.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.953+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6806b7e0 con 0x7f0a68104c50 2024-06-03T01:10:20.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.953+0000 7f0a66000640 1 -- 172.21.15.94:0/512426748 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a58005500 con 0x7f0a68104c50 2024-06-03T01:10:20.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.953+0000 7f0a66000640 1 -- 172.21.15.94:0/512426748 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58004e00 con 0x7f0a68104c50 2024-06-03T01:10:20.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.953+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6806bd30 con 0x7f0a68104c50 2024-06-03T01:10:20.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.955+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6810ad80 con 0x7f0a68104c50 2024-06-03T01:10:20.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.958+0000 7f0a66000640 1 -- 172.21.15.94:0/512426748 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f0a58004fa0 con 0x7f0a68104c50 2024-06-03T01:10:20.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.960+0000 7f0a66000640 1 -- 172.21.15.94:0/512426748 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f0a580ce8f0 con 0x7f0a68104c50 2024-06-03T01:10:20.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:20.968+0000 7f0a66000640 1 -- 172.21.15.94:0/512426748 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a5809b120 con 0x7f0a68104c50 2024-06-03T01:10:21.016 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:20 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:20.676+0000 7f052e92e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-03T01:10:21.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.196+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f0a680625a0 con 0x7f0a68104c50 2024-06-03T01:10:21.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.201+0000 7f0a66000640 1 -- 172.21.15.94:0/512426748 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v34)=0 v34) ==== 138+0+0 (secure 0 0 0) 0x7f0a5809f420 con 0x7f0a68104c50 2024-06-03T01:10:21.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68104c50 msgr2=0x7f0a6819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68104c50 0x7f0a6819a2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800d770 tx=0x7f0a5800dc30 comp rx=0 tx=0).stop 2024-06-03T01:10:21.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 shutdown_connections 2024-06-03T01:10:21.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a68105610 0x7f0a681a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a68104c50 0x7f0a6819a2b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 --2- 172.21.15.94:0/512426748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a68103a60 0x7f0a68199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 >> 172.21.15.94:0/512426748 conn(0x7f0a680fefd0 msgr2=0x7f0a68107f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:21.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 shutdown_connections 2024-06-03T01:10:21.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.204+0000 7f0a6f000640 1 -- 172.21.15.94:0/512426748 wait complete. 2024-06-03T01:10:21.222 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.033+0000 7f0484200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.033+0000 7f0484200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.033+0000 7f0484200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.033+0000 7f0484200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.033+0000 7f0484200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.034+0000 7f0480600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.034+0000 7f0480600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.282 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.034+0000 7f0480600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.283 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.034+0000 7f0480600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.283 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:10:21.034+0000 7f0480600700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:10:21.283 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-03T01:10:21.283 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:10:21.283 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: Active manager daemon x restarted 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: Activating manager daemon x 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: mgrmap e21: x(active, starting, since 0.0200224s), standbys: y, z 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:10:21.284 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:10:21.285 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3527570999' entity='client.admin' 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: Manager daemon x is now available 2024-06-03T01:10:21.286 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:10:21.287 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:10:21.287 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:10:21.287 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T01:10:21.287 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T01:10:21.287 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:21 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/512426748' entity='client.admin' 2024-06-03T01:10:21.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 -- 172.21.15.94:0/2038124194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 msgr2=0x7f2674104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 --2- 172.21.15.94:0/2038124194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674104040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2664009900 tx=0x7f266402a410 comp rx=0 tx=0).stop 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 -- 172.21.15.94:0/2038124194 shutdown_connections 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 --2- 172.21.15.94:0/2038124194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26741057d0 0x7f267410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 --2- 172.21.15.94:0/2038124194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2674104e10 0x7f2674105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 --2- 172.21.15.94:0/2038124194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674104040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 -- 172.21.15.94:0/2038124194 >> 172.21.15.94:0/2038124194 conn(0x7f26740ff190 msgr2=0x7f26741015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 -- 172.21.15.94:0/2038124194 shutdown_connections 2024-06-03T01:10:21.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.395+0000 7f267a400640 1 -- 172.21.15.94:0/2038124194 wait complete. 2024-06-03T01:10:21.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.396+0000 7f267a400640 1 Processor -- start 2024-06-03T01:10:21.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.396+0000 7f267a400640 1 -- start start 2024-06-03T01:10:21.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f267a400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:21.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f267a400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2674104e10 0x7f267419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f267a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26741057d0 0x7f26741a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f267a400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f267410f890 con 0x7f2674104e10 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f267a400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f267410f710 con 0x7f26741057d0 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f267a400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f267410fa10 con 0x7f2674103c20 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f2673400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f2672a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2674104e10 0x7f267419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f2673e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26741057d0 0x7f26741a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f2673400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:45534/0 (socket says 172.21.15.94:45534) 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f2673e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26741057d0 0x7f26741a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41802/0 (socket says 172.21.15.94:41802) 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f2673400640 1 -- 172.21.15.94:0/3655850175 learned_addr learned my addr 172.21.15.94:0/3655850175 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.397+0000 7f2672a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2674104e10 0x7f267419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48324/0 (socket says 172.21.15.94:48324) 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2673400640 1 -- 172.21.15.94:0/3655850175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26741057d0 msgr2=0x7f26741a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2673400640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26741057d0 0x7f26741a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2673400640 1 -- 172.21.15.94:0/3655850175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2674104e10 msgr2=0x7f267419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2673400640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2674104e10 0x7f267419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2673400640 1 -- 172.21.15.94:0/3655850175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26741a1ac0 con 0x7f2674103c20 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2673e00640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26741057d0 0x7f26741a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2672a00640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2674104e10 0x7f267419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f2673400640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674199ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2664031610 tx=0x7f2664004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f266be00640 1 -- 172.21.15.94:0/3655850175 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266403f030 con 0x7f2674103c20 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f266be00640 1 -- 172.21.15.94:0/3655850175 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2664003ad0 con 0x7f2674103c20 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741a1d50 con 0x7f2674103c20 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f266be00640 1 -- 172.21.15.94:0/3655850175 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664035070 con 0x7f2674103c20 2024-06-03T01:10:21.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.398+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741a21d0 con 0x7f2674103c20 2024-06-03T01:10:21.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.399+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267410af40 con 0x7f2674103c20 2024-06-03T01:10:21.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.401+0000 7f266be00640 1 -- 172.21.15.94:0/3655850175 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f2664042030 con 0x7f2674103c20 2024-06-03T01:10:21.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.403+0000 7f266be00640 1 -- 172.21.15.94:0/3655850175 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f26640ef480 con 0x7f2674103c20 2024-06-03T01:10:21.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.407+0000 7f266be00640 1 -- 172.21.15.94:0/3655850175 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f26640bbcb0 con 0x7f2674103c20 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: Active manager daemon x restarted 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: Activating manager daemon x 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: mgrmap e21: x(active, starting, since 0.0200224s), standbys: y, z 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T01:10:21.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T01:10:21.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:10:21.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3527570999' entity='client.admin' 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: Manager daemon x is now available 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T01:10:21.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T01:10:21.538 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:21 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/512426748' entity='client.admin' 2024-06-03T01:10:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-03T01:10:21.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: Active manager daemon x restarted 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: Activating manager daemon x 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: osdmap e101: 12 total, 12 up, 12 in 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: mgrmap e21: x(active, starting, since 0.0200224s), standbys: y, z 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:10:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-03T01:10:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3527570999' entity='client.admin' 2024-06-03T01:10:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: Manager daemon x is now available 2024-06-03T01:10:21.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:10:21.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:10:21.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-03T01:10:21.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T01:10:21.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-03T01:10:21.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:21 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/512426748' entity='client.admin' 2024-06-03T01:10:21.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.607+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f267402d2e0 con 0x7f2674103c20 2024-06-03T01:10:21.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.613+0000 7f266be00640 1 -- 172.21.15.94:0/3655850175 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v35)=0 v35) ==== 133+0+0 (secure 0 0 0) 0x7f26640bffb0 con 0x7f2674103c20 2024-06-03T01:10:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 msgr2=0x7f2674199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674199ea0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2664031610 tx=0x7f2664004270 comp rx=0 tx=0).stop 2024-06-03T01:10:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 shutdown_connections 2024-06-03T01:10:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26741057d0 0x7f26741a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2674104e10 0x7f267419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 --2- 172.21.15.94:0/3655850175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2674103c20 0x7f2674199ea0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 >> 172.21.15.94:0/3655850175 conn(0x7f26740ff190 msgr2=0x7f2674108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:21.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 shutdown_connections 2024-06-03T01:10:21.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.616+0000 7f267a400640 1 -- 172.21.15.94:0/3655850175 wait complete. 2024-06-03T01:10:21.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-06-03T01:10:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 -- 172.21.15.94:0/1719553115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c1098c0 msgr2=0x7ff76c10bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 --2- 172.21.15.94:0/1719553115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c1098c0 0x7ff76c10bd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff764009ca0 tx=0x7ff76402a8c0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 -- 172.21.15.94:0/1719553115 shutdown_connections 2024-06-03T01:10:21.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 --2- 172.21.15.94:0/1719553115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c1098c0 0x7ff76c10bd50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 --2- 172.21.15.94:0/1719553115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff76c074400 0x7ff76c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 --2- 172.21.15.94:0/1719553115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c0759e0 0x7ff76c073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 -- 172.21.15.94:0/1719553115 >> 172.21.15.94:0/1719553115 conn(0x7ff76c0fcf90 msgr2=0x7ff76c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:21.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 -- 172.21.15.94:0/1719553115 shutdown_connections 2024-06-03T01:10:21.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.813+0000 7ff773c00640 1 -- 172.21.15.94:0/1719553115 wait complete. 2024-06-03T01:10:21.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.814+0000 7ff773c00640 1 Processor -- start 2024-06-03T01:10:21.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.814+0000 7ff773c00640 1 -- start start 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.814+0000 7ff773c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff76c074400 0x7ff76c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff773c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c0759e0 0x7ff76c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff773c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1098c0 0x7ff76c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff773c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c10f840 con 0x7ff76c0759e0 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff773c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c10f6c0 con 0x7ff76c074400 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff773c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c10f9c0 con 0x7ff76c1098c0 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff76be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c0759e0 0x7ff76c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff76be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c0759e0 0x7ff76c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48340/0 (socket says 172.21.15.94:48340) 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff771200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff76c074400 0x7ff76c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff76be00640 1 -- 172.21.15.94:0/2203934157 learned_addr learned my addr 172.21.15.94:0/2203934157 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff771c00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1098c0 0x7ff76c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff771200640 1 -- 172.21.15.94:0/2203934157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1098c0 msgr2=0x7ff76c1a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff771200640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1098c0 0x7ff76c1a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff771200640 1 -- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c0759e0 msgr2=0x7ff76c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff771200640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c0759e0 0x7ff76c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff771200640 1 -- 172.21.15.94:0/2203934157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c1a19d0 con 0x7ff76c074400 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.815+0000 7ff76be00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c0759e0 0x7ff76c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.816+0000 7ff771200640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff76c074400 0x7ff76c199db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff76000e9f0 tx=0x7ff76000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.816+0000 7ff769600640 1 -- 172.21.15.94:0/2203934157 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76000cd80 con 0x7ff76c074400 2024-06-03T01:10:21.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.816+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c1a1cc0 con 0x7ff76c074400 2024-06-03T01:10:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.816+0000 7ff769600640 1 -- 172.21.15.94:0/2203934157 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7600035f0 con 0x7ff76c074400 2024-06-03T01:10:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.816+0000 7ff769600640 1 -- 172.21.15.94:0/2203934157 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760010670 con 0x7ff76c074400 2024-06-03T01:10:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.816+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c1a21c0 con 0x7ff76c074400 2024-06-03T01:10:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.817+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76c1085f0 con 0x7ff76c074400 2024-06-03T01:10:21.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.820+0000 7ff769600640 1 -- 172.21.15.94:0/2203934157 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7ff7600037a0 con 0x7ff76c074400 2024-06-03T01:10:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.821+0000 7ff769600640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff7540a2a40 0x7ff7540a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:21.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.821+0000 7ff769600640 1 -- 172.21.15.94:0/2203934157 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7ff7600cf3e0 con 0x7ff76c074400 2024-06-03T01:10:21.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.821+0000 7ff76be00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff7540a2a40 0x7ff7540a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:21.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.822+0000 7ff76be00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff7540a2a40 0x7ff7540a4f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7580095f0 tx=0x7ff758005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:21.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:21.826+0000 7ff769600640 1 -- 172.21.15.94:0/2203934157 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff76009c170 con 0x7ff76c074400 2024-06-03T01:10:22.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.054+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 --> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff76c061de0 con 0x7ff7540a2a40 2024-06-03T01:10:22.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.126+0000 7ff769600640 1 -- 172.21.15.94:0/2203934157 <== mgr.15084 v2:172.21.15.94:6800/2692604051 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff76c061de0 con 0x7ff7540a2a40 2024-06-03T01:10:22.126 INFO:teuthology.orchestra.run.smithi094.stdout:Schedule set for path / 2024-06-03T01:10:22.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff7540a2a40 msgr2=0x7ff7540a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff7540a2a40 0x7ff7540a4f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7580095f0 tx=0x7ff758005f90 comp rx=0 tx=0).stop 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff76c074400 msgr2=0x7ff76c199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff76c074400 0x7ff76c199db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff76000e9f0 tx=0x7ff76000eeb0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 shutdown_connections 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff7540a2a40 0x7ff7540a4f00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff76c1098c0 0x7ff76c1a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff76c0759e0 0x7ff76c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 --2- 172.21.15.94:0/2203934157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff76c074400 0x7ff76c199db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 >> 172.21.15.94:0/2203934157 conn(0x7ff76c0fcf90 msgr2=0x7ff76c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 shutdown_connections 2024-06-03T01:10:22.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.128+0000 7ff773c00640 1 -- 172.21.15.94:0/2203934157 wait complete. 2024-06-03T01:10:22.146 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-06-03T01:10:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.313+0000 7fc60c600640 1 -- 172.21.15.94:0/1861919393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc604102bf0 msgr2=0x7fc604103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.313+0000 7fc60c600640 1 --2- 172.21.15.94:0/1861919393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc604102bf0 0x7fc604103070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009900 tx=0x7fc5f002a3f0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.313+0000 7fc60c600640 1 -- 172.21.15.94:0/1861919393 shutdown_connections 2024-06-03T01:10:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.313+0000 7fc60c600640 1 --2- 172.21.15.94:0/1861919393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6041035b0 0x7fc604109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.313+0000 7fc60c600640 1 --2- 172.21.15.94:0/1861919393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc604102bf0 0x7fc604103070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.313+0000 7fc60c600640 1 --2- 172.21.15.94:0/1861919393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc604101a00 0x7fc604101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.313+0000 7fc60c600640 1 -- 172.21.15.94:0/1861919393 >> 172.21.15.94:0/1861919393 conn(0x7fc6040fcf90 msgr2=0x7fc6040ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.314+0000 7fc60c600640 1 -- 172.21.15.94:0/1861919393 shutdown_connections 2024-06-03T01:10:22.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.314+0000 7fc60c600640 1 -- 172.21.15.94:0/1861919393 wait complete. 2024-06-03T01:10:22.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.314+0000 7fc60c600640 1 Processor -- start 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60c600640 1 -- start start 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60c600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc604101a00 0x7fc604199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60c600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc604102bf0 0x7fc60419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60c600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6041035b0 0x7fc6041a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60c600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d6d0 con 0x7fc6041035b0 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60c600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d550 con 0x7fc604102bf0 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60c600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d850 con 0x7fc604101a00 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc609200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc604102bf0 0x7fc60419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc60a600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6041035b0 0x7fc6041a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc609200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc604102bf0 0x7fc60419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41822/0 (socket says 172.21.15.94:41822) 2024-06-03T01:10:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc609200640 1 -- 172.21.15.94:0/1105666031 learned_addr learned my addr 172.21.15.94:0/1105666031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.315+0000 7fc609c00640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc604101a00 0x7fc604199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc609200640 1 -- 172.21.15.94:0/1105666031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc604101a00 msgr2=0x7fc604199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc609200640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc604101a00 0x7fc604199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc609200640 1 -- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6041035b0 msgr2=0x7fc6041a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc609200640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6041035b0 0x7fc6041a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc609200640 1 -- 172.21.15.94:0/1105666031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6041a1a70 con 0x7fc604102bf0 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc60a600640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6041035b0 0x7fc6041a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc609200640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc604102bf0 0x7fc60419a390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0031440 tx=0x7fc5f0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc602000640 1 -- 172.21.15.94:0/1105666031 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f003f030 con 0x7fc604102bf0 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6041a1d00 con 0x7fc604102bf0 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc602000640 1 -- 172.21.15.94:0/1105666031 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f0003ce0 con 0x7fc604102bf0 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc602000640 1 -- 172.21.15.94:0/1105666031 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0035070 con 0x7fc604102bf0 2024-06-03T01:10:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.316+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6041a21e0 con 0x7fc604102bf0 2024-06-03T01:10:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.317+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6041a2410 con 0x7fc604102bf0 2024-06-03T01:10:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.319+0000 7fc602000640 1 -- 172.21.15.94:0/1105666031 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fc5f0042030 con 0x7fc604102bf0 2024-06-03T01:10:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.321+0000 7fc602000640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5ec0a2a40 0x7fc5ec0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.321+0000 7fc602000640 1 -- 172.21.15.94:0/1105666031 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fc5f00ef550 con 0x7fc604102bf0 2024-06-03T01:10:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.321+0000 7fc609c00640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5ec0a2a40 0x7fc5ec0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.321+0000 7fc609c00640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5ec0a2a40 0x7fc5ec0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc604102a50 tx=0x7fc5f8005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:22.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.325+0000 7fc602000640 1 -- 172.21.15.94:0/1105666031 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc5f00bbcd0 con 0x7fc604102bf0 2024-06-03T01:10:22.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.551+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 --> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc6040752f0 con 0x7fc5ec0a2a40 2024-06-03T01:10:22.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.624+0000 7fc602000640 1 -- 172.21.15.94:0/1105666031 <== mgr.15084 v2:172.21.15.94:6800/2692604051 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc6040752f0 con 0x7fc5ec0a2a40 2024-06-03T01:10:22.624 INFO:teuthology.orchestra.run.smithi094.stdout:Retention added to path / 2024-06-03T01:10:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5ec0a2a40 msgr2=0x7fc5ec0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5ec0a2a40 0x7fc5ec0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc604102a50 tx=0x7fc5f8005f90 comp rx=0 tx=0).stop 2024-06-03T01:10:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc604102bf0 msgr2=0x7fc60419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc604102bf0 0x7fc60419a390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0031440 tx=0x7fc5f0032ce0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 shutdown_connections 2024-06-03T01:10:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5ec0a2a40 0x7fc5ec0a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6041035b0 0x7fc6041a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc604102bf0 0x7fc60419a390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 --2- 172.21.15.94:0/1105666031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc604101a00 0x7fc604199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 >> 172.21.15.94:0/1105666031 conn(0x7fc6040fcf90 msgr2=0x7fc604073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:22.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 shutdown_connections 2024-06-03T01:10:22.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.626+0000 7fc60c600640 1 -- 172.21.15.94:0/1105666031 wait complete. 2024-06-03T01:10:22.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-03T01:10:22.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' 2024-06-03T01:10:22.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: mgrmap e22: x(active, since 1.07348s), standbys: y, z 2024-06-03T01:10:22.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: [03/Jun/2024:01:10:21] ENGINE Bus STARTING 2024-06-03T01:10:22.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:22.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: [03/Jun/2024:01:10:21] ENGINE Serving on https://172.21.15.94:7150 2024-06-03T01:10:22.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: [03/Jun/2024:01:10:21] ENGINE Serving on http://172.21.15.94:8765 2024-06-03T01:10:22.692 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: [03/Jun/2024:01:10:21] ENGINE Bus STARTED 2024-06-03T01:10:22.693 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:22 smithi094 ceph-mon[32119]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 -- 172.21.15.94:0/2234529857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 msgr2=0x7fe70c1048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 --2- 172.21.15.94:0/2234529857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c1048d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009900 tx=0x7fe6fc02a3f0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 -- 172.21.15.94:0/2234529857 shutdown_connections 2024-06-03T01:10:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 --2- 172.21.15.94:0/2234529857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c107740 0x7fe70c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 --2- 172.21.15.94:0/2234529857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70c104e10 0x7fe70c107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 --2- 172.21.15.94:0/2234529857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c1048d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 -- 172.21.15.94:0/2234529857 >> 172.21.15.94:0/2234529857 conn(0x7fe70c0fbe70 msgr2=0x7fe70c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 -- 172.21.15.94:0/2234529857 shutdown_connections 2024-06-03T01:10:22.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.813+0000 7fe711600640 1 -- 172.21.15.94:0/2234529857 wait complete. 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 Processor -- start 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 -- start start 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c104e10 0x7fe70c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70c107740 0x7fe70c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10d6a0 con 0x7fe70c107740 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10d520 con 0x7fe70c104e10 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe711600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c10d820 con 0x7fe70c1024e0 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.814+0000 7fe70aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:45570/0 (socket says 172.21.15.94:45570) 2024-06-03T01:10:22.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 -- 172.21.15.94:0/2079751867 learned_addr learned my addr 172.21.15.94:0/2079751867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70a000640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c104e10 0x7fe70c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70b400640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70c107740 0x7fe70c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 -- 172.21.15.94:0/2079751867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c104e10 msgr2=0x7fe70c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c104e10 0x7fe70c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 -- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70c107740 msgr2=0x7fe70c1a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70c107740 0x7fe70c1a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 -- 172.21.15.94:0/2079751867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c1a1ad0 con 0x7fe70c1024e0 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.815+0000 7fe70aa00640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c199eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009230 tx=0x7fe6fc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.816+0000 7fe703400640 1 -- 172.21.15.94:0/2079751867 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc03f030 con 0x7fe70c1024e0 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.816+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c1a1d60 con 0x7fe70c1024e0 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.816+0000 7fe703400640 1 -- 172.21.15.94:0/2079751867 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6fc003ce0 con 0x7fe70c1024e0 2024-06-03T01:10:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.816+0000 7fe703400640 1 -- 172.21.15.94:0/2079751867 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc035070 con 0x7fe70c1024e0 2024-06-03T01:10:22.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.816+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a21e0 con 0x7fe70c1024e0 2024-06-03T01:10:22.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.817+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c109230 con 0x7fe70c1024e0 2024-06-03T01:10:22.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.819+0000 7fe703400640 1 -- 172.21.15.94:0/2079751867 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fe6fc042030 con 0x7fe70c1024e0 2024-06-03T01:10:22.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.821+0000 7fe703400640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe6f00a2ac0 0x7fe6f00a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.821+0000 7fe703400640 1 -- 172.21.15.94:0/2079751867 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe6fc0ef8e0 con 0x7fe70c1024e0 2024-06-03T01:10:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.821+0000 7fe70a000640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe6f00a2ac0 0x7fe6f00a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:22.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.821+0000 7fe70a000640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe6f00a2ac0 0x7fe6f00a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80095f0 tx=0x7fe6f8005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:22.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:22.823+0000 7fe703400640 1 -- 172.21.15.94:0/2079751867 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe6fc0bc020 con 0x7fe70c1024e0 2024-06-03T01:10:23.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' 2024-06-03T01:10:23.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: mgrmap e22: x(active, since 1.07348s), standbys: y, z 2024-06-03T01:10:23.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: [03/Jun/2024:01:10:21] ENGINE Bus STARTING 2024-06-03T01:10:23.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:23.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: [03/Jun/2024:01:10:21] ENGINE Serving on https://172.21.15.94:7150 2024-06-03T01:10:23.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: [03/Jun/2024:01:10:21] ENGINE Serving on http://172.21.15.94:8765 2024-06-03T01:10:23.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: [03/Jun/2024:01:10:21] ENGINE Bus STARTED 2024-06-03T01:10:23.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:22 smithi107 ceph-mon[36622]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:23.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.072+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 --> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe70c061de0 con 0x7fe6f00a2ac0 2024-06-03T01:10:23.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.081+0000 7fe703400640 1 -- 172.21.15.94:0/2079751867 <== mgr.15084 v2:172.21.15.94:6800/2692604051 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fe70c061de0 con 0x7fe6f00a2ac0 2024-06-03T01:10:23.081 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-03T00:00:00", "created": "2024-06-03T01:10:22", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-06-03T01:10:23.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe6f00a2ac0 msgr2=0x7fe6f00a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:23.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe6f00a2ac0 0x7fe6f00a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80095f0 tx=0x7fe6f8005f90 comp rx=0 tx=0).stop 2024-06-03T01:10:23.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 msgr2=0x7fe70c199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:23.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c199eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009230 tx=0x7fe6fc0042f0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 shutdown_connections 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe6f00a2ac0 0x7fe6f00a4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70c107740 0x7fe70c1a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c104e10 0x7fe70c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 --2- 172.21.15.94:0/2079751867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c1024e0 0x7fe70c199eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 >> 172.21.15.94:0/2079751867 conn(0x7fe70c0fbe70 msgr2=0x7fe70c101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 shutdown_connections 2024-06-03T01:10:23.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.083+0000 7fe711600640 1 -- 172.21.15.94:0/2079751867 wait complete. 2024-06-03T01:10:23.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-03T01:10:23.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' 2024-06-03T01:10:23.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: mgrmap e22: x(active, since 1.07348s), standbys: y, z 2024-06-03T01:10:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: [03/Jun/2024:01:10:21] ENGINE Bus STARTING 2024-06-03T01:10:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: [03/Jun/2024:01:10:21] ENGINE Serving on https://172.21.15.94:7150 2024-06-03T01:10:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: [03/Jun/2024:01:10:21] ENGINE Serving on http://172.21.15.94:8765 2024-06-03T01:10:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: [03/Jun/2024:01:10:21] ENGINE Bus STARTED 2024-06-03T01:10:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:22 smithi179 ceph-mon[37041]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.292+0000 7fd3d6a00640 1 -- 172.21.15.94:0/331610757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0101830 msgr2=0x7fd3d0101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.292+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/331610757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0101830 0x7fd3d0101c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009900 tx=0x7fd3c402a410 comp rx=0 tx=0).stop 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.293+0000 7fd3d6a00640 1 -- 172.21.15.94:0/331610757 shutdown_connections 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.293+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/331610757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d01033e0 0x7fd3d0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.293+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/331610757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d0102a20 0x7fd3d0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.293+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/331610757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0101830 0x7fd3d0101c50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.293+0000 7fd3d6a00640 1 -- 172.21.15.94:0/331610757 >> 172.21.15.94:0/331610757 conn(0x7fd3d00fcda0 msgr2=0x7fd3d00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.293+0000 7fd3d6a00640 1 -- 172.21.15.94:0/331610757 shutdown_connections 2024-06-03T01:10:23.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.293+0000 7fd3d6a00640 1 -- 172.21.15.94:0/331610757 wait complete. 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.294+0000 7fd3d6a00640 1 Processor -- start 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.294+0000 7fd3d6a00640 1 -- start start 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.294+0000 7fd3d6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d0101830 0x7fd3d0075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d6a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0102a20 0x7fd3d0073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d01033e0 0x7fd3d00742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010d4f0 con 0x7fd3d0101830 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010d370 con 0x7fd3d01033e0 2024-06-03T01:10:23.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d6a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d010d670 con 0x7fd3d0102a20 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d0101830 0x7fd3d0075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0102a20 0x7fd3d0073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0102a20 0x7fd3d0073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:45610/0 (socket says 172.21.15.94:45610) 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d0101830 0x7fd3d0075750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48418/0 (socket says 172.21.15.94:48418) 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d01033e0 0x7fd3d00742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 -- 172.21.15.94:0/2808585119 learned_addr learned my addr 172.21.15.94:0/2808585119 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 -- 172.21.15.94:0/2808585119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d01033e0 msgr2=0x7fd3d00742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d01033e0 0x7fd3d00742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 -- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d0101830 msgr2=0x7fd3d0075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:23.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d0101830 0x7fd3d0075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.295+0000 7fd3d4c00640 1 -- 172.21.15.94:0/2808585119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d0074b60 con 0x7fd3d0102a20 2024-06-03T01:10:23.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.296+0000 7fd3d4c00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0102a20 0x7fd3d0073d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e9f0 tx=0x7fd3bc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:23.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.296+0000 7fd3ce000640 1 -- 172.21.15.94:0/2808585119 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00cd80 con 0x7fd3d0102a20 2024-06-03T01:10:23.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.296+0000 7fd3ce000640 1 -- 172.21.15.94:0/2808585119 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3bc005500 con 0x7fd3d0102a20 2024-06-03T01:10:23.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.296+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01a1c30 con 0x7fd3d0102a20 2024-06-03T01:10:23.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.296+0000 7fd3ce000640 1 -- 172.21.15.94:0/2808585119 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc010690 con 0x7fd3d0102a20 2024-06-03T01:10:23.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.296+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a2130 con 0x7fd3d0102a20 2024-06-03T01:10:23.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.297+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d0108b50 con 0x7fd3d0102a20 2024-06-03T01:10:23.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.299+0000 7fd3ce000640 1 -- 172.21.15.94:0/2808585119 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fd3bc0046c0 con 0x7fd3d0102a20 2024-06-03T01:10:23.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.301+0000 7fd3ce000640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd3b80a2970 0x7fd3b80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:23.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.301+0000 7fd3ce000640 1 -- 172.21.15.94:0/2808585119 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fd3bc0cf580 con 0x7fd3d0102a20 2024-06-03T01:10:23.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.301+0000 7fd3d5600640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd3b80a2970 0x7fd3b80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:23.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.302+0000 7fd3d5600640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd3b80a2970 0x7fd3b80a4e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3c402aa00 tx=0x7fd3c40315b0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:23.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.304+0000 7fd3ce000640 1 -- 172.21.15.94:0/2808585119 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd3bc09c310 con 0x7fd3d0102a20 2024-06-03T01:10:23.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.539+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 --> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd3d0061de0 con 0x7fd3b80a2970 2024-06-03T01:10:23.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.560+0000 7fd3ce000640 1 -- 172.21.15.94:0/2808585119 <== mgr.15084 v2:172.21.15.94:6800/2692604051 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd3d0061de0 con 0x7fd3b80a2970 2024-06-03T01:10:23.560 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd3b80a2970 msgr2=0x7fd3b80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd3b80a2970 0x7fd3b80a4e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3c402aa00 tx=0x7fd3c40315b0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0102a20 msgr2=0x7fd3d0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0102a20 0x7fd3d0073d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e9f0 tx=0x7fd3bc00edf0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 shutdown_connections 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd3b80a2970 0x7fd3b80a4e30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d01033e0 0x7fd3d00742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3d0102a20 0x7fd3d0073d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 --2- 172.21.15.94:0/2808585119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3d0101830 0x7fd3d0075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 >> 172.21.15.94:0/2808585119 conn(0x7fd3d00fcda0 msgr2=0x7fd3d0105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 shutdown_connections 2024-06-03T01:10:23.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:10:23.564+0000 7fd3d6a00640 1 -- 172.21.15.94:0/2808585119 wait complete. 2024-06-03T01:10:23.582 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-06-03T01:10:23.653 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-06-03T01:10:23.664 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-03T01:10:23.674 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:10:23.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-03T01:10:23.678 INFO:teuthology.run_tasks:Running task workunit... 2024-06-03T01:10:23.692 INFO:tasks.workunit:Pulling workunits from ref dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-03T01:10:23.692 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-03T01:10:23.692 INFO:tasks.workunit:timeout=3h 2024-06-03T01:10:23.692 INFO:tasks.workunit:cleanup=True 2024-06-03T01:10:23.693 DEBUG:teuthology.orchestra.run.smithi094:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-06-03 01:06:39.207654976 +0000 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-06-03 01:06:39.207654976 +0000 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-06-03 01:10:18.276457692 +0000 2024-06-03T01:10:23.735 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-06-03 01:06:39.207654976 +0000 2024-06-03T01:10:23.736 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-03T01:10:23.736 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-03T01:10:23.849 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-03T01:10:23.893 INFO:tasks.workunit.client.0.smithi094.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-03T01:10:24.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:24.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:24.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: from='client.25300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:24.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-06-03T01:10:24.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:23 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: from='client.25300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-06-03T01:10:24.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:23 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:24.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: from='client.25300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:24.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-06-03T01:10:24.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:23 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:25.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: Unable to set osd_memory_target on smithi107 to 184723660: error parsing value: Value '184723660' is below minimum 939524096 2024-06-03T01:10:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-06-03T01:10:25.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:24 smithi179 ceph-mon[37041]: Unable to set osd_memory_target on smithi179 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-06-03T01:10:25.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:25.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: Unable to set osd_memory_target on smithi107 to 184723660: error parsing value: Value '184723660' is below minimum 939524096 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.285 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-06-03T01:10:25.286 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:24 smithi107 ceph-mon[36622]: Unable to set osd_memory_target on smithi179 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-06-03T01:10:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:10:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: Unable to set osd_memory_target on smithi107 to 184723660: error parsing value: Value '184723660' is below minimum 939524096 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-06-03T01:10:25.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:24 smithi094 ceph-mon[32119]: Unable to set osd_memory_target on smithi179 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-06-03T01:10:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-06-03T01:10:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:26.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:26.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:26.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:26 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-06-03T01:10:26.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:26.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:26.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:26.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:26 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-06-03T01:10:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:10:26.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:26.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:26 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:27 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-06-03T01:10:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:27 smithi107 ceph-mon[36622]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-06-03T01:10:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:27 smithi107 ceph-mon[36622]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-03T01:10:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:27 smithi107 ceph-mon[36622]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T01:10:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:27 smithi107 ceph-mon[36622]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T01:10:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:27 smithi107 ceph-mon[36622]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:27.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:27 smithi107 ceph-mon[36622]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:27 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-06-03T01:10:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:27 smithi094 ceph-mon[32119]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-06-03T01:10:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:27 smithi094 ceph-mon[32119]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-03T01:10:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:27 smithi094 ceph-mon[32119]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T01:10:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:27 smithi094 ceph-mon[32119]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T01:10:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:27 smithi094 ceph-mon[32119]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:27.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:27 smithi094 ceph-mon[32119]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:27 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-06-03T01:10:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:27 smithi179 ceph-mon[37041]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-06-03T01:10:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:27 smithi179 ceph-mon[37041]: Updating smithi094:/etc/ceph/ceph.conf 2024-06-03T01:10:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:27 smithi179 ceph-mon[37041]: Updating smithi107:/etc/ceph/ceph.conf 2024-06-03T01:10:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:27 smithi179 ceph-mon[37041]: Updating smithi179:/etc/ceph/ceph.conf 2024-06-03T01:10:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:27 smithi179 ceph-mon[37041]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:27.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:27 smithi179 ceph-mon[37041]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:28.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:28.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:28 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:28 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:10:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-03T01:10:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: Updating smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: Updating smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:28.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:28.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:28 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:29.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:29.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:29 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:29.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:29.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:10:29.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:29.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:29.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:29 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: Updating smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:10:29.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:29 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:10:30.684 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:30 smithi107 ceph-mon[36622]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-06-03T01:10:30.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:30 smithi094 ceph-mon[32119]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-06-03T01:10:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:30 smithi179 ceph-mon[37041]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-06-03T01:10:32.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:32 smithi107 ceph-mon[36622]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-06-03T01:10:32.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:32 smithi094 ceph-mon[32119]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-06-03T01:10:32.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:32 smithi179 ceph-mon[37041]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-06-03T01:10:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:34 smithi179 ceph-mon[37041]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-06-03T01:10:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:34 smithi107 ceph-mon[36622]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-06-03T01:10:34.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:34 smithi094 ceph-mon[32119]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-06-03T01:10:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:36 smithi107 ceph-mon[36622]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-06-03T01:10:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:36 smithi094 ceph-mon[32119]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-06-03T01:10:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:36 smithi179 ceph-mon[37041]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 -- 172.21.15.94:0/2579476812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 msgr2=0x7f06a410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 --2- 172.21.15.94:0/2579476812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a410e220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f069c009ca0 tx=0x7f069c02a950 comp rx=0 tx=0).stop 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 -- 172.21.15.94:0/2579476812 shutdown_connections 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 --2- 172.21.15.94:0/2579476812 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a410e220 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 --2- 172.21.15.94:0/2579476812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06a4103320 0x7f06a410b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 --2- 172.21.15.94:0/2579476812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a4102a00 0x7f06a4102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 -- 172.21.15.94:0/2579476812 >> 172.21.15.94:0/2579476812 conn(0x7f06a40fc1c0 msgr2=0x7f06a40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 -- 172.21.15.94:0/2579476812 shutdown_connections 2024-06-03T01:10:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.917+0000 7f06a3200640 1 -- 172.21.15.94:0/2579476812 wait complete. 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.918+0000 7f06a3200640 1 Processor -- start 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.918+0000 7f06a3200640 1 -- start start 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06a4102a00 0x7f06a4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a3200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a4103320 0x7f06a41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4111b70 con 0x7f06a410bdd0 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41119f0 con 0x7f06a4102a00 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a3200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4111cf0 con 0x7f06a4103320 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06a4102a00 0x7f06a4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57704/0 (socket says 172.21.15.94:57704) 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f06a2800640 1 -- 172.21.15.94:0/2393728097 learned_addr learned my addr 172.21.15.94:0/2393728097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.919+0000 7f069be00640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a4103320 0x7f06a41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a2800640 1 -- 172.21.15.94:0/2393728097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a4103320 msgr2=0x7f06a41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a2800640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a4103320 0x7f06a41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a2800640 1 -- 172.21.15.94:0/2393728097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06a4102a00 msgr2=0x7f06a4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a2800640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06a4102a00 0x7f06a4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a2800640 1 -- 172.21.15.94:0/2393728097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a419d030 con 0x7f06a410bdd0 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a1e00640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06a4102a00 0x7f06a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a2800640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a419c900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f069c004af0 tx=0x7f069c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f069aa00640 1 -- 172.21.15.94:0/2393728097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c034040 con 0x7f06a410bdd0 2024-06-03T01:10:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a419d2c0 con 0x7f06a410bdd0 2024-06-03T01:10:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f069aa00640 1 -- 172.21.15.94:0/2393728097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069c030920 con 0x7f06a410bdd0 2024-06-03T01:10:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f069aa00640 1 -- 172.21.15.94:0/2393728097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c004670 con 0x7f06a410bdd0 2024-06-03T01:10:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.920+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41a4bb0 con 0x7f06a410bdd0 2024-06-03T01:10:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.925+0000 7f06ab81c740 1 Processor -- start 2024-06-03T01:10:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.925+0000 7f069aa00640 1 -- 172.21.15.94:0/2393728097 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f069c037020 con 0x7f06a410bdd0 2024-06-03T01:10:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.925+0000 7f06ab81c740 1 -- start start 2024-06-03T01:10:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff1760770 0x556ff175ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff175f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff1770780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06ab81c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556ff16c3420 con 0x556ff175f200 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06ab81c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556ff166ac50 con 0x556ff176e390 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06ab81c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556ff16bf3e0 con 0x556ff1760770 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff175f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff175f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57720/0 (socket says 172.21.15.94:57720) 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 -- 172.21.15.94:0/4103243542 learned_addr learned my addr 172.21.15.94:0/4103243542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f069be00640 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff1760770 0x556ff175ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f069be00640 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff1770780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 -- 172.21.15.94:0/4103243542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff1760770 msgr2=0x556ff175ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff1760770 0x556ff175ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 -- 172.21.15.94:0/4103243542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 msgr2=0x556ff1770780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff1770780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.926+0000 7f06a1e00640 1 -- 172.21.15.94:0/4103243542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff13d7210 con 0x556ff175f200 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.927+0000 7f06a1e00640 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff175f680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f06940115c0 tx=0x7f0694011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83a04add12cdc66 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.927+0000 7f069aa00640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f06880a2e00 0x7f06880a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.927+0000 7f0693000640 1 -- 172.21.15.94:0/4103243542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069400f640 con 0x556ff175f200 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.927+0000 7f069aa00640 1 -- 172.21.15.94:0/2393728097 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f069c0f0470 con 0x7f06a410bdd0 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.927+0000 7f0693000640 1 -- 172.21.15.94:0/4103243542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069400fcc0 con 0x556ff175f200 2024-06-03T01:10:36.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.927+0000 7f0693000640 1 -- 172.21.15.94:0/4103243542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694019d30 con 0x556ff175f200 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.927+0000 7f069be00640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f06880a2e00 0x7f06880a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 -- 172.21.15.94:0/4103243542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 msgr2=0x556ff175f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f069be00640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f06880a2e00 0x7f06880a52c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x556ff175c980 tx=0x7f068c0046c0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff175f680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f06940115c0 tx=0x7f0694011a80 comp rx=0 tx=0).stop 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 -- 172.21.15.94:0/4103243542 shutdown_connections 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff1770780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff175f680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 --2- 172.21.15.94:0/4103243542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff1760770 0x556ff175ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 -- 172.21.15.94:0/4103243542 >> 172.21.15.94:0/4103243542 conn(0x556ff1727f80 msgr2=0x556ff175b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 -- 172.21.15.94:0/4103243542 shutdown_connections 2024-06-03T01:10:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.928+0000 7f06ab81c740 1 -- 172.21.15.94:0/4103243542 wait complete. 2024-06-03T01:10:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 -- start start 2024-06-03T01:10:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 Processor -- start 2024-06-03T01:10:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff175f200 0x556ff176b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff176b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff176bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556ff1771160 con 0x556ff1760770 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556ff167e0b0 con 0x556ff176e390 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.929+0000 7f06ab81c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556ff17712e0 con 0x556ff175f200 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f06ab81c740 10 client.0 _ll_register_callbacks cb 0x556ff1669f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff176b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff176b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57730/0 (socket says 172.21.15.94:57730) 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 -- 172.21.15.94:0/3286370436 learned_addr learned my addr 172.21.15.94:0/3286370436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f06a2800640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff176bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f06a1e00640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff175f200 0x556ff176b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 -- 172.21.15.94:0/3286370436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff175f200 msgr2=0x556ff176b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff175f200 0x556ff176b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 -- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 msgr2=0x556ff176bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff176bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f069be00640 1 -- 172.21.15.94:0/3286370436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff14a3620 con 0x556ff1760770 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.930+0000 7f06a2800640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff176bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f069be00640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff176b880 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f068c004520 tx=0x7f068c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aba303c4081a687 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f0687400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f0687400640 1 -- 172.21.15.94:0/3286370436 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c019030 con 0x556ff1760770 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f0687400640 1 -- 172.21.15.94:0/3286370436 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f068c022720 con 0x556ff1760770 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f069400fcc0 con 0x556ff1760770 2024-06-03T01:10:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f0687400640 1 -- 172.21.15.94:0/3286370436 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f068c034050 con 0x556ff1760770 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f06ab81c740 10 client.15186 fetch_fsmap learned FSMap version 28 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556ff177d4f0 con 0x556ff1760770 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.931+0000 7f0687400640 1 -- 172.21.15.94:0/3286370436 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069400fcc0 con 0x556ff1760770 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f0687400640 1 -- 172.21.15.94:0/3286370436 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f068c02bac0 con 0x556ff1760770 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f06ab81c740 10 client.15186 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f06ab81c740 10 client.15186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f06ab81c740 20 client.15186 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f06ab81c740 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff177e9b0 0x556ff1780dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f06ab81c740 4 client.15186 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff16d0b50 con 0x556ff177e9b0 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.932+0000 7f06a1e00640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff177e9b0 0x556ff1780dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.933+0000 7f06a1e00640 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff177e9b0 0x556ff1780dc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5184f4ce99ce27 server_cookie=6ed9849959bc9ad2 in_seq=0 out_seq=0 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.933+0000 7f0687400640 10 client.15186 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.933+0000 7f0687400640 1 -- 172.21.15.94:0/3286370436 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556ff16d0b50 con 0x556ff177e9b0 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.933+0000 7f0687400640 10 client.15186 handle_command_reply: tid=0 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.933+0000 7f06ab81c740 1 client.15186 shutdown 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 msgr2=0x556ff176b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff176b880 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f068c004520 tx=0x7f068c005f90 comp rx=0 tx=0).stop 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 shutdown_connections 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff177e9b0 0x556ff1780dc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff176e390 0x556ff176bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff176b880 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 --2- 172.21.15.94:0/3286370436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff175f200 0x556ff176b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f0687400640 0 client.15186 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 >> 172.21.15.94:0/3286370436 conn(0x556ff1727f80 msgr2=0x556ff17615f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 shutdown_connections 2024-06-03T01:10:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 1 -- 172.21.15.94:0/3286370436 wait complete. 2024-06-03T01:10:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.934+0000 7f06ab81c740 20 client.15186 trim_cache size 0 max 16384 2024-06-03T01:10:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.950+0000 7f06ab81c740 1 Processor -- start 2024-06-03T01:10:36.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06ab81c740 1 -- start start 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff175f200 0x556ff1761470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff175b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff175b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06ab81c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41ac950 con 0x556ff1760770 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06ab81c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41ae490 con 0x556ff175f200 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06ab81c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41ad020 con 0x556ff176e390 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f069be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff175b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06a1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff175f200 0x556ff1761470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f069be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff175b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57742/0 (socket says 172.21.15.94:57742) 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f069be00640 1 -- 172.21.15.94:0/1363225158 learned_addr learned my addr 172.21.15.94:0/1363225158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06a2800640 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff175b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06a1e00640 1 -- 172.21.15.94:0/1363225158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 msgr2=0x556ff175b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.951+0000 7f06a1e00640 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff175b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f06a1e00640 1 -- 172.21.15.94:0/1363225158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 msgr2=0x556ff175b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f06a1e00640 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff175b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f06a1e00640 1 -- 172.21.15.94:0/1363225158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff1769890 con 0x556ff175f200 2024-06-03T01:10:36.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f069be00640 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff175b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f06a1e00640 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff175f200 0x556ff1761470 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f069401b040 tx=0x7f0694009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6beedf456f4ed18c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f0692600640 1 -- 172.21.15.94:0/1363225158 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069402c070 con 0x556ff175f200 2024-06-03T01:10:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f0692600640 1 -- 172.21.15.94:0/1363225158 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694019e20 con 0x556ff175f200 2024-06-03T01:10:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.952+0000 7f0692600640 1 -- 172.21.15.94:0/1363225158 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069400f8a0 con 0x556ff175f200 2024-06-03T01:10:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 -- 172.21.15.94:0/1363225158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff175f200 msgr2=0x556ff1761470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff175f200 0x556ff1761470 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f069401b040 tx=0x7f0694009310 comp rx=0 tx=0).stop 2024-06-03T01:10:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 -- 172.21.15.94:0/1363225158 shutdown_connections 2024-06-03T01:10:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff175b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff1760770 0x556ff175b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 --2- 172.21.15.94:0/1363225158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff175f200 0x556ff1761470 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 -- 172.21.15.94:0/1363225158 >> 172.21.15.94:0/1363225158 conn(0x556ff1727f80 msgr2=0x556ff1760d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 -- 172.21.15.94:0/1363225158 shutdown_connections 2024-06-03T01:10:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 -- 172.21.15.94:0/1363225158 wait complete. 2024-06-03T01:10:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.953+0000 7f06ab81c740 1 -- start start 2024-06-03T01:10:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 1 Processor -- start 2024-06-03T01:10:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff1776870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff1760770 0x556ff1776db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff1767d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41addc0 con 0x556ff175f200 2024-06-03T01:10:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41ad6f0 con 0x556ff1760770 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41aa930 con 0x556ff176e390 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06ab81c740 10 client.0 _ll_register_callbacks cb 0x7f068c009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff1776870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff1776870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57744/0 (socket says 172.21.15.94:57744) 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06a1e00640 1 -- 172.21.15.94:0/2579843293 learned_addr learned my addr 172.21.15.94:0/2579843293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f06a2800640 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff1767d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.954+0000 7f069be00640 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff1760770 0x556ff1776db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f06a1e00640 1 -- 172.21.15.94:0/2579843293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 msgr2=0x556ff1767d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f06a1e00640 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff1767d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f06a1e00640 1 -- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff1760770 msgr2=0x556ff1776db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f06a1e00640 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff1760770 0x556ff1776db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f06a1e00640 1 -- 172.21.15.94:0/2579843293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff1768590 con 0x556ff175f200 2024-06-03T01:10:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f06a1e00640 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff1776870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0694013040 tx=0x7f06940051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f013c8be85754af6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f0687400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:10:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f0687400640 1 -- 172.21.15.94:0/2579843293 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069402c070 con 0x556ff175f200 2024-06-03T01:10:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f0687400640 1 -- 172.21.15.94:0/2579843293 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f06940055c0 con 0x556ff175f200 2024-06-03T01:10:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.955+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0694019e20 con 0x556ff175f200 2024-06-03T01:10:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f0687400640 1 -- 172.21.15.94:0/2579843293 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0694019650 con 0x556ff175f200 2024-06-03T01:10:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 10 client.15192 fetch_fsmap learned FSMap version 28 2024-06-03T01:10:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556ff17e9270 con 0x556ff175f200 2024-06-03T01:10:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f0687400640 1 -- 172.21.15.94:0/2579843293 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694019e20 con 0x556ff175f200 2024-06-03T01:10:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f0687400640 1 -- 172.21.15.94:0/2579843293 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f06940035f0 con 0x556ff175f200 2024-06-03T01:10:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 10 client.15192 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:10:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 10 client.15192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:10:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 20 client.15192 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff17e9740 0x556ff17ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 4 client.15192 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:10:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.956+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ff16caea0 con 0x556ff17e9740 2024-06-03T01:10:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.957+0000 7f069be00640 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff17e9740 0x556ff17ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.957+0000 7f069be00640 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff17e9740 0x556ff17ebb30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28db16276bab4b20 server_cookie=3ac06c5bf3132628 in_seq=0 out_seq=0 2024-06-03T01:10:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.957+0000 7f0687400640 10 client.15192 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.970+0000 7f0687400640 1 -- 172.21.15.94:0/2579843293 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556ff16caea0 con 0x556ff17e9740 2024-06-03T01:10:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.970+0000 7f0687400640 10 client.15192 handle_command_reply: tid=0 2024-06-03T01:10:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.970+0000 7f06ab81c740 1 client.15192 shutdown 2024-06-03T01:10:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.970+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 msgr2=0x556ff1776870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.970+0000 7f06ab81c740 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff1776870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0694013040 tx=0x7f06940051c0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 shutdown_connections 2024-06-03T01:10:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ff176e390 0x556ff1767d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556ff17e9740 0x556ff17ebb30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ff1760770 0x556ff1776db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 --2- 172.21.15.94:0/2579843293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556ff175f200 0x556ff1776870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f0687400640 0 client.15192 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 >> 172.21.15.94:0/2579843293 conn(0x556ff1727f80 msgr2=0x556ff176bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 shutdown_connections 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 1 -- 172.21.15.94:0/2579843293 wait complete. 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.971+0000 7f06ab81c740 20 client.15192 trim_cache size 0 max 16384 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:10:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:10:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f06880a2e00 msgr2=0x7f06880a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f06880a2e00 0x7f06880a52c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x556ff175c980 tx=0x7f068c0046c0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 msgr2=0x7f06a419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a419c900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f069c004af0 tx=0x7f069c02f3a0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 shutdown_connections 2024-06-03T01:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f06880a2e00 0x7f06880a52c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a410bdd0 0x7f06a419c900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06a4103320 0x7f06a41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 --2- 172.21.15.94:0/2393728097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06a4102a00 0x7f06a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 >> 172.21.15.94:0/2393728097 conn(0x7f06a40fc1c0 msgr2=0x7f06a40fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 shutdown_connections 2024-06-03T01:10:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:36.972+0000 7f06a3200640 1 -- 172.21.15.94:0/2393728097 wait complete. 2024-06-03T01:10:38.652 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:38 smithi094 ceph-mon[32119]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-06-03T01:10:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:38 smithi107 ceph-mon[36622]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-06-03T01:10:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:38 smithi179 ceph-mon[37041]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-06-03T01:10:40.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:40 smithi107 ceph-mon[36622]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-06-03T01:10:40.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:40 smithi094 ceph-mon[32119]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-06-03T01:10:40.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:40 smithi179 ceph-mon[37041]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-06-03T01:10:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:42 smithi107 ceph-mon[36622]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:10:42.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:42 smithi094 ceph-mon[32119]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:10:42.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:42 smithi179 ceph-mon[37041]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:10:44.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:44 smithi107 ceph-mon[36622]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-06-03T01:10:44.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:44 smithi094 ceph-mon[32119]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-06-03T01:10:44.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:44 smithi179 ceph-mon[37041]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-06-03T01:10:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:46 smithi107 ceph-mon[36622]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:46.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:46 smithi094 ceph-mon[32119]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:46.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:46 smithi179 ceph-mon[37041]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:48 smithi107 ceph-mon[36622]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:48 smithi094 ceph-mon[32119]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:48 smithi179 ceph-mon[37041]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 -- 172.21.15.94:0/3513991404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a50108bb0 msgr2=0x7f7a5010cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 --2- 172.21.15.94:0/3513991404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a50108bb0 0x7f7a5010cbf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009900 tx=0x7f7a3802a3f0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 -- 172.21.15.94:0/3513991404 shutdown_connections 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 --2- 172.21.15.94:0/3513991404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a5010d2d0 0x7f7a50114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 --2- 172.21.15.94:0/3513991404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a50108bb0 0x7f7a5010cbf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 --2- 172.21.15.94:0/3513991404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a50108200 0x7f7a501085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 -- 172.21.15.94:0/3513991404 >> 172.21.15.94:0/3513991404 conn(0x7f7a50075f30 msgr2=0x7f7a50078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 -- 172.21.15.94:0/3513991404 shutdown_connections 2024-06-03T01:10:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.114+0000 7f7a55000640 1 -- 172.21.15.94:0/3513991404 wait complete. 2024-06-03T01:10:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 Processor -- start 2024-06-03T01:10:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 -- start start 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a50108200 0x7f7a501a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5010d2d0 0x7f7a501a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a5019e5f0 0x7f7a5019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50118390 con 0x7f7a5019e5f0 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50118210 con 0x7f7a50108200 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.116+0000 7f7a55000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50118510 con 0x7f7a5010d2d0 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5010d2d0 0x7f7a501a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5010d2d0 0x7f7a501a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:57154/0 (socket says 172.21.15.94:57154) 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 -- 172.21.15.94:0/3599005091 learned_addr learned my addr 172.21.15.94:0/3599005091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4e800640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a5019e5f0 0x7f7a5019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4de00640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a50108200 0x7f7a501a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 -- 172.21.15.94:0/3599005091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a50108200 msgr2=0x7f7a501a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a50108200 0x7f7a501a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 -- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a5019e5f0 msgr2=0x7f7a5019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a5019e5f0 0x7f7a5019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 -- 172.21.15.94:0/3599005091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5019f110 con 0x7f7a5010d2d0 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.117+0000 7f7a4d400640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5010d2d0 0x7f7a501a4980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a3802a910 tx=0x7f7a38003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.118+0000 7f7a46a00640 1 -- 172.21.15.94:0/3599005091 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3803f070 con 0x7f7a5010d2d0 2024-06-03T01:10:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.118+0000 7f7a46a00640 1 -- 172.21.15.94:0/3599005091 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a38031c90 con 0x7f7a5010d2d0 2024-06-03T01:10:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.118+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5019f3a0 con 0x7f7a5010d2d0 2024-06-03T01:10:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.118+0000 7f7a46a00640 1 -- 172.21.15.94:0/3599005091 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38035030 con 0x7f7a5010d2d0 2024-06-03T01:10:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.118+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a501ab5f0 con 0x7f7a5010d2d0 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.121+0000 7f7a57e38740 1 Processor -- start 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.121+0000 7f7a57e38740 1 -- start start 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.121+0000 7f7a46a00640 1 -- 172.21.15.94:0/3599005091 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f7a38042070 con 0x7f7a5010d2d0 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.121+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dcab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.121+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dcb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.121+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a57e38740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556fb4d2f420 con 0x556fb4dcc5b0 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a57e38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556fb4cd6c50 con 0x556fb4dcb040 2024-06-03T01:10:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a57e38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556fb4d2b3e0 con 0x556fb4dd5dc0 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dcab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dcab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37930/0 (socket says 172.21.15.94:37930) 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4e800640 1 -- 172.21.15.94:0/3625670863 learned_addr learned my addr 172.21.15.94:0/3625670863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4de00640 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dcb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4d400640 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4de00640 1 -- 172.21.15.94:0/3625670863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 msgr2=0x556fb4dd81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4de00640 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4de00640 1 -- 172.21.15.94:0/3625670863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 msgr2=0x556fb4dcab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4de00640 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dcab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4de00640 1 -- 172.21.15.94:0/3625670863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb4a43210 con 0x556fb4dcb040 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.122+0000 7f7a4e800640 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dcab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.123+0000 7f7a4de00640 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dcb4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009ba0 tx=0x7f7a400117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acd1098aec67af7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.123+0000 7f7a3f400640 1 -- 172.21.15.94:0/3625670863 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40012450 con 0x556fb4dcb040 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.123+0000 7f7a3f400640 1 -- 172.21.15.94:0/3625670863 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a40012ad0 con 0x556fb4dcb040 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.123+0000 7f7a3f400640 1 -- 172.21.15.94:0/3625670863 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000fb90 con 0x556fb4dcb040 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.123+0000 7f7a46a00640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a340a2ec0 0x7f7a340a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.123+0000 7f7a46a00640 1 -- 172.21.15.94:0/3599005091 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f7a380effe0 con 0x7f7a5010d2d0 2024-06-03T01:10:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.123+0000 7f7a4e800640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a340a2ec0 0x7f7a340a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 -- 172.21.15.94:0/3625670863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 msgr2=0x556fb4dcb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dcb4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009ba0 tx=0x7f7a400117b0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 -- 172.21.15.94:0/3625670863 shutdown_connections 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dcb4c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 --2- 172.21.15.94:0/3625670863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dcab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 -- 172.21.15.94:0/3625670863 >> 172.21.15.94:0/3625670863 conn(0x556fb4d93f80 msgr2=0x556fb4dc7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 -- 172.21.15.94:0/3625670863 shutdown_connections 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a57e38740 1 -- 172.21.15.94:0/3625670863 wait complete. 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.124+0000 7f7a4e800640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a340a2ec0 0x7f7a340a5380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009fd0 tx=0x7f7a48009500 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 -- start start 2024-06-03T01:10:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 Processor -- start 2024-06-03T01:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dd2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556fb4dd8b90 con 0x556fb4dcc5b0 2024-06-03T01:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556fb4cea0b0 con 0x556fb4dcb040 2024-06-03T01:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556fb4dd8d10 con 0x556fb4dd5dc0 2024-06-03T01:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a57e38740 10 client.0 _ll_register_callbacks cb 0x556fb4cd5f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.125+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37932/0 (socket says 172.21.15.94:37932) 2024-06-03T01:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 -- 172.21.15.94:0/593090627 learned_addr learned my addr 172.21.15.94:0/593090627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4de00640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dd2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4e800640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 -- 172.21.15.94:0/593090627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 msgr2=0x556fb4dd3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 -- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 msgr2=0x556fb4dd2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dd2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 -- 172.21.15.94:0/593090627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb4b0f620 con 0x556fb4dcc5b0 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4de00640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dd2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a4d400640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd3330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a380bbd30 tx=0x7f7a38009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf820a5b8718313 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a33e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a33e00640 1 -- 172.21.15.94:0/593090627 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38049050 con 0x556fb4dcc5b0 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.126+0000 7f7a33e00640 1 -- 172.21.15.94:0/593090627 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f7a380f3cd0 con 0x556fb4dcc5b0 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a40012ad0 con 0x556fb4dcc5b0 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a33e00640 1 -- 172.21.15.94:0/593090627 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a380bbe00 con 0x556fb4dcc5b0 2024-06-03T01:10:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 10 client.15204 fetch_fsmap learned FSMap version 28 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fb4de4f00 con 0x556fb4dcc5b0 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a33e00640 1 -- 172.21.15.94:0/593090627 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a40012ad0 con 0x556fb4dcc5b0 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a33e00640 1 -- 172.21.15.94:0/593090627 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a380ed5c0 con 0x556fb4dcc5b0 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 10 client.15204 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 10 client.15204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 20 client.15204 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4de6360 0x556fb4de87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 4 client.15204 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.127+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fb4d3cb50 con 0x556fb4de6360 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.128+0000 7f7a4de00640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4de6360 0x556fb4de87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.128+0000 7f7a4de00640 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4de6360 0x556fb4de87f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77572de59bb1777f server_cookie=58c9f689838b01eb in_seq=0 out_seq=0 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.128+0000 7f7a33e00640 10 client.15204 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.129+0000 7f7a33e00640 1 -- 172.21.15.94:0/593090627 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556fb4d3cb50 con 0x556fb4de6360 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.129+0000 7f7a33e00640 10 client.15204 handle_command_reply: tid=0 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.129+0000 7f7a57e38740 1 client.15204 shutdown 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 msgr2=0x556fb4dd3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd3330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7a380bbd30 tx=0x7f7a38009260 comp rx=0 tx=0).stop 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 shutdown_connections 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd3330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4de6360 0x556fb4de87f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 --2- 172.21.15.94:0/593090627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcb040 0x556fb4dd2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 >> 172.21.15.94:0/593090627 conn(0x556fb4d93f80 msgr2=0x556fb4dcd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a33e00640 0 client.15204 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 shutdown_connections 2024-06-03T01:10:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 1 -- 172.21.15.94:0/593090627 wait complete. 2024-06-03T01:10:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.130+0000 7f7a57e38740 20 client.15204 trim_cache size 0 max 16384 2024-06-03T01:10:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 Processor -- start 2024-06-03T01:10:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 -- start start 2024-06-03T01:10:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcb040 0x556fb4dcd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcc5b0 0x556fb4dc6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dd5dc0 0x556fb4dc74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501b52c0 con 0x556fb4dd5dc0 2024-06-03T01:10:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501b6e00 con 0x556fb4dcc5b0 2024-06-03T01:10:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.147+0000 7f7a57e38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501b5990 con 0x556fb4dcb040 2024-06-03T01:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcc5b0 0x556fb4dc6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dd5dc0 0x556fb4dc74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcc5b0 0x556fb4dc6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:47032/0 (socket says 172.21.15.94:47032) 2024-06-03T01:10:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 -- 172.21.15.94:0/1471537143 learned_addr learned my addr 172.21.15.94:0/1471537143 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4de00640 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcb040 0x556fb4dcd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 -- 172.21.15.94:0/1471537143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcb040 msgr2=0x556fb4dcd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcb040 0x556fb4dcd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 -- 172.21.15.94:0/1471537143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dd5dc0 msgr2=0x556fb4dc74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dd5dc0 0x556fb4dc74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 -- 172.21.15.94:0/1471537143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb4dc2ff0 con 0x556fb4dcc5b0 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.148+0000 7f7a4d400640 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcc5b0 0x556fb4dc6f90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a380bbfc0 tx=0x7f7a380f0370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3573d073b72657a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a3ea00640 1 -- 172.21.15.94:0/1471537143 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38049070 con 0x556fb4dcc5b0 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a3ea00640 1 -- 172.21.15.94:0/1471537143 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a380f3800 con 0x556fb4dcc5b0 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a3ea00640 1 -- 172.21.15.94:0/1471537143 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a380bb990 con 0x556fb4dcc5b0 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 -- 172.21.15.94:0/1471537143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcc5b0 msgr2=0x556fb4dc6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcc5b0 0x556fb4dc6f90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a380bbfc0 tx=0x7f7a380f0370 comp rx=0 tx=0).stop 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 -- 172.21.15.94:0/1471537143 shutdown_connections 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dd5dc0 0x556fb4dc74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dcc5b0 0x556fb4dc6f90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1471537143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcb040 0x556fb4dcd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 -- 172.21.15.94:0/1471537143 >> 172.21.15.94:0/1471537143 conn(0x556fb4d93f80 msgr2=0x556fb4dccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.149+0000 7f7a57e38740 1 -- 172.21.15.94:0/1471537143 shutdown_connections 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.150+0000 7f7a57e38740 1 -- 172.21.15.94:0/1471537143 wait complete. 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.150+0000 7f7a57e38740 1 -- start start 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.150+0000 7f7a57e38740 1 Processor -- start 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcb040 0x556fb4dde680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a57e38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a57e38740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501b6730 con 0x556fb4dcb040 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a57e38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501b6060 con 0x556fb4dd5dc0 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a57e38740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501b32a0 con 0x556fb4dcc5b0 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a57e38740 10 client.0 _ll_register_callbacks cb 0x7f7a38005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcb040 0x556fb4dde680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcb040 0x556fb4dde680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37948/0 (socket says 172.21.15.94:37948) 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 -- 172.21.15.94:0/1078939731 learned_addr learned my addr 172.21.15.94:0/1078939731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4d400640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4e800640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 -- 172.21.15.94:0/1078939731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcc5b0 msgr2=0x556fb4dd99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 -- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dd5dc0 msgr2=0x556fb4dd9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4de00640 1 -- 172.21.15.94:0/1078939731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb4dda6e0 con 0x556fb4dcb040 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.151+0000 7f7a4d400640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a4de00640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcb040 0x556fb4dde680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000a010 tx=0x7f7a4000f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1187ea219e1925d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a33e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a33e00640 1 -- 172.21.15.94:0/1078939731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000fb60 con 0x556fb4dcb040 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a33e00640 1 -- 172.21.15.94:0/1078939731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f7a4000fd00 con 0x556fb4dcb040 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a380f3800 con 0x556fb4dcb040 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a33e00640 1 -- 172.21.15.94:0/1078939731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a400050b0 con 0x556fb4dcb040 2024-06-03T01:10:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a57e38740 10 client.15210 fetch_fsmap learned FSMap version 28 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.152+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fb4e50ff0 con 0x556fb4dcb040 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a33e00640 1 -- 172.21.15.94:0/1078939731 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a380f3800 con 0x556fb4dcb040 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a33e00640 1 -- 172.21.15.94:0/1078939731 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a40003bd0 con 0x556fb4dcb040 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a57e38740 10 client.15210 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a57e38740 10 client.15210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a57e38740 20 client.15210 populate_metadata read hostname 'smithi094' 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4e51570 0x556fb4e53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a57e38740 4 client.15210 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fb4d36ea0 con 0x556fb4e51570 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.153+0000 7f7a4d400640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4e51570 0x556fb4e53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.154+0000 7f7a4d400640 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4e51570 0x556fb4e53960 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c70afcb62885226 server_cookie=a3ce32e7dda9e2a1 in_seq=0 out_seq=0 2024-06-03T01:10:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.154+0000 7f7a33e00640 10 client.15210 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a33e00640 1 -- 172.21.15.94:0/1078939731 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556fb4d36ea0 con 0x556fb4e51570 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a33e00640 10 client.15210 handle_command_reply: tid=0 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 client.15210 shutdown 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcb040 msgr2=0x556fb4dde680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcb040 0x556fb4dde680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000a010 tx=0x7f7a4000f7e0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 shutdown_connections 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556fb4e51570 0x556fb4e53960 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556fb4dd5dc0 0x556fb4dd9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556fb4dcc5b0 0x556fb4dd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a33e00640 0 client.15210 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:10:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.157+0000 7f7a57e38740 1 --2- 172.21.15.94:0/1078939731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fb4dcb040 0x556fb4dde680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 >> 172.21.15.94:0/1078939731 conn(0x556fb4d93f80 msgr2=0x556fb4dcd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 shutdown_connections 2024-06-03T01:10:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a57e38740 1 -- 172.21.15.94:0/1078939731 wait complete. 2024-06-03T01:10:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a57e38740 20 client.15210 trim_cache size 0 max 16384 2024-06-03T01:10:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:10:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:10:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:10:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a340a2ec0 msgr2=0x7f7a340a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a55000640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a340a2ec0 0x7f7a340a5380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009fd0 tx=0x7f7a48009500 comp rx=0 tx=0).stop 2024-06-03T01:10:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5010d2d0 msgr2=0x7f7a501a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:10:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.158+0000 7f7a55000640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5010d2d0 0x7f7a501a4980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a3802a910 tx=0x7f7a38003b20 comp rx=0 tx=0).stop 2024-06-03T01:10:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 shutdown_connections 2024-06-03T01:10:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a340a2ec0 0x7f7a340a5380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a5019e5f0 0x7f7a5019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5010d2d0 0x7f7a501a4980 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 --2- 172.21.15.94:0/3599005091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a50108200 0x7f7a501a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 >> 172.21.15.94:0/3599005091 conn(0x7f7a50075f30 msgr2=0x7f7a50076950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 shutdown_connections 2024-06-03T01:10:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:10:50.159+0000 7f7a55000640 1 -- 172.21.15.94:0/3599005091 wait complete. 2024-06-03T01:10:50.728 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:50 smithi094 ceph-mon[32119]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-06-03T01:10:50.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:50 smithi107 ceph-mon[36622]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-06-03T01:10:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:50 smithi179 ceph-mon[37041]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.5 KiB/s wr, 17 op/s 2024-06-03T01:10:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:52 smithi107 ceph-mon[36622]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:52 smithi094 ceph-mon[32119]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:52 smithi179 ceph-mon[37041]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-03T01:10:54.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:54 smithi107 ceph-mon[36622]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-06-03T01:10:54.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:54 smithi094 ceph-mon[32119]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-06-03T01:10:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:54 smithi179 ceph-mon[37041]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-06-03T01:10:56.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:56 smithi107 ceph-mon[36622]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:10:56.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:56 smithi094 ceph-mon[32119]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:10:56.851 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:56 smithi179 ceph-mon[37041]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:10:58.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:10:58 smithi107 ceph-mon[36622]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:10:58.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:10:58 smithi094 ceph-mon[32119]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:10:58.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:10:58 smithi179 ceph-mon[37041]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:00 smithi107 ceph-mon[36622]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:00.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:00 smithi094 ceph-mon[32119]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:00.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:11:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:11:00.648+0000 7f045a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:11:00.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:11:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:11:00.648+0000 7f045a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:11:00.850 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:11:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:11:00.648+0000 7f045a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:11:00.850 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:11:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:11:00.648+0000 7f045a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:11:00.850 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:11:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:11:00.648+0000 7f045a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-03T01:11:00.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:00 smithi179 ceph-mon[37041]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:02 smithi107 ceph-mon[36622]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:02 smithi107 ceph-mon[36622]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-06-03T01:11:02.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:02 smithi094 ceph-mon[32119]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:02.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:02 smithi094 ceph-mon[32119]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-06-03T01:11:02.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:02 smithi179 ceph-mon[37041]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:02.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:02 smithi179 ceph-mon[37041]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 -- 172.21.15.94:0/4212194601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72641047c0 msgr2=0x7f7264108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 --2- 172.21.15.94:0/4212194601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72641047c0 0x7f7264108800 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7258009900 tx=0x7f725802a3f0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 -- 172.21.15.94:0/4212194601 shutdown_connections 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 --2- 172.21.15.94:0/4212194601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7264108ee0 0x7f7264110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 --2- 172.21.15.94:0/4212194601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72641047c0 0x7f7264108800 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 --2- 172.21.15.94:0/4212194601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7264103e10 0x7f72641041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 -- 172.21.15.94:0/4212194601 >> 172.21.15.94:0/4212194601 conn(0x7f72640ff6d0 msgr2=0x7f7264101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 -- 172.21.15.94:0/4212194601 shutdown_connections 2024-06-03T01:11:03.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.302+0000 7f726c000640 1 -- 172.21.15.94:0/4212194601 wait complete. 2024-06-03T01:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.304+0000 7f726c000640 1 Processor -- start 2024-06-03T01:11:03.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.304+0000 7f726c000640 1 -- start start 2024-06-03T01:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f726c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7264103e10 0x7f726419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f726c000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7264108ee0 0x7f726419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f726c000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726419eda0 0x7f726419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f726c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7264113fe0 con 0x7f7264103e10 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f726c000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7264113e60 con 0x7f7264108ee0 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f726c000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7264114160 con 0x7f726419eda0 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7269600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7264103e10 0x7f726419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7268c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7264108ee0 0x7f726419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7269600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7264103e10 0x7f726419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44856/0 (socket says 172.21.15.94:44856) 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7268c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7264108ee0 0x7f726419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33826/0 (socket says 172.21.15.94:33826) 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7269600640 1 -- 172.21.15.94:0/1271185459 learned_addr learned my addr 172.21.15.94:0/1271185459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f726a000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726419eda0 0x7f726419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7268c00640 1 -- 172.21.15.94:0/1271185459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726419eda0 msgr2=0x7f726419f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7268c00640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726419eda0 0x7f726419f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7268c00640 1 -- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7264103e10 msgr2=0x7f726419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7268c00640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7264103e10 0x7f726419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.305+0000 7f7268c00640 1 -- 172.21.15.94:0/1271185459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726419f890 con 0x7f7264108ee0 2024-06-03T01:11:03.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.306+0000 7f7268c00640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7264108ee0 0x7f726419aa20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f725802a910 tx=0x7f7258003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.306+0000 7f7262000640 1 -- 172.21.15.94:0/1271185459 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725803f070 con 0x7f7264108ee0 2024-06-03T01:11:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.306+0000 7f7262000640 1 -- 172.21.15.94:0/1271185459 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7258031c90 con 0x7f7264108ee0 2024-06-03T01:11:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.306+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726419fb20 con 0x7f7264108ee0 2024-06-03T01:11:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.306+0000 7f7262000640 1 -- 172.21.15.94:0/1271185459 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258035030 con 0x7f7264108ee0 2024-06-03T01:11:03.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.306+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72641a7340 con 0x7f7264108ee0 2024-06-03T01:11:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.309+0000 7f726ef99740 1 Processor -- start 2024-06-03T01:11:03.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.309+0000 7f7262000640 1 -- 172.21.15.94:0/1271185459 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f7258042070 con 0x7f7264108ee0 2024-06-03T01:11:03.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726ef99740 1 -- start start 2024-06-03T01:11:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726ef99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726ef99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb89e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726ef99740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726ef99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cb8949420 con 0x557cb89e65b0 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726ef99740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cb88f0c50 con 0x557cb89e5040 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726ef99740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557cb89453e0 con 0x557cb89f85d0 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44862/0 (socket says 172.21.15.94:44862) 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f7269600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb89e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f726a000640 1 -- 172.21.15.94:0/240551355 learned_addr learned my addr 172.21.15.94:0/240551355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.310+0000 7f7268c00640 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f726a000640 1 -- 172.21.15.94:0/240551355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 msgr2=0x557cb89fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f726a000640 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f726a000640 1 -- 172.21.15.94:0/240551355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 msgr2=0x557cb89e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f726a000640 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb89e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f726a000640 1 -- 172.21.15.94:0/240551355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb865d210 con 0x557cb89e65b0 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f7269600640 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb89e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f726a000640 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e4b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f725c009f50 tx=0x7f725c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e619c775b48886a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f7262000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f724c0a2f40 0x7f724c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f7252600640 1 -- 172.21.15.94:0/240551355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c004d20 con 0x557cb89e65b0 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f7252600640 1 -- 172.21.15.94:0/240551355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f725c004050 con 0x557cb89e65b0 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f7252600640 1 -- 172.21.15.94:0/240551355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c005280 con 0x557cb89e65b0 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.311+0000 7f7262000640 1 -- 172.21.15.94:0/1271185459 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f72580f01b0 con 0x7f7264108ee0 2024-06-03T01:11:03.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f7269600640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f724c0a2f40 0x7f724c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 -- 172.21.15.94:0/240551355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 msgr2=0x557cb89e4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e4b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f725c009f50 tx=0x7f725c0118d0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f7269600640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f724c0a2f40 0x7f724c0a5400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x557cb89e7d10 tx=0x7f7254005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 -- 172.21.15.94:0/240551355 shutdown_connections 2024-06-03T01:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb89e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 --2- 172.21.15.94:0/240551355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e4b00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 -- 172.21.15.94:0/240551355 >> 172.21.15.94:0/240551355 conn(0x557cb89adf80 msgr2=0x557cb89e1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 -- 172.21.15.94:0/240551355 shutdown_connections 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.312+0000 7f726ef99740 1 -- 172.21.15.94:0/240551355 wait complete. 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.313+0000 7f726ef99740 1 -- start start 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.313+0000 7f726ef99740 1 Processor -- start 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.313+0000 7f726ef99740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726ef99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e65b0 0x557cb89f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726ef99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89f85d0 0x557cb89f6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726ef99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cb89fb3a0 con 0x557cb89f85d0 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726ef99740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cb89040b0 con 0x557cb89e65b0 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726ef99740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557cb89fb520 con 0x557cb89e5040 2024-06-03T01:11:03.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726ef99740 10 client.0 _ll_register_callbacks cb 0x557cb88eff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89f85d0 0x557cb89f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7268c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e65b0 0x557cb89f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89f85d0 0x557cb89f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44868/0 (socket says 172.21.15.94:44868) 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7268c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e65b0 0x557cb89f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33850/0 (socket says 172.21.15.94:33850) 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726a000640 1 -- 172.21.15.94:0/689772205 learned_addr learned my addr 172.21.15.94:0/689772205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7269600640 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7268c00640 1 -- 172.21.15.94:0/689772205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 msgr2=0x557cb89f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7268c00640 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7268c00640 1 -- 172.21.15.94:0/689772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89f85d0 msgr2=0x557cb89f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7268c00640 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89f85d0 0x557cb89f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f7268c00640 1 -- 172.21.15.94:0/689772205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb8729620 con 0x557cb89e65b0 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.314+0000 7f726a000640 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89f85d0 0x557cb89f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:03.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f7268c00640 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e65b0 0x557cb89f5ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72580ed3f0 tx=0x7f7258031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6159edd428b3e762 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f7246a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f7246a00640 1 -- 172.21.15.94:0/689772205 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258049070 con 0x557cb89e65b0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f725c004050 con 0x557cb89e65b0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f7246a00640 1 -- 172.21.15.94:0/689772205 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f72580dc0b0 con 0x557cb89e65b0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f7246a00640 1 -- 172.21.15.94:0/689772205 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72580bc070 con 0x557cb89e65b0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f726ef99740 10 client.25349 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cb8a076d0 con 0x557cb89e65b0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.315+0000 7f7246a00640 1 -- 172.21.15.94:0/689772205 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f725c004050 con 0x557cb89e65b0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f7246a00640 1 -- 172.21.15.94:0/689772205 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72580c0120 con 0x557cb89e65b0 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f726ef99740 10 client.25349 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f726ef99740 10 client.25349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f726ef99740 20 client.25349 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:03.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f726ef99740 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a08c00 0x557cb8a0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f726ef99740 4 client.25349 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:11:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cb8956b50 con 0x557cb8a08c00 2024-06-03T01:11:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.316+0000 7f7269600640 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a08c00 0x557cb8a0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.317+0000 7f7269600640 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a08c00 0x557cb8a0b040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e36eae836305fb server_cookie=4a9e2cb082988fa8 in_seq=0 out_seq=0 2024-06-03T01:11:03.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.317+0000 7f7246a00640 10 client.25349 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f7246a00640 1 -- 172.21.15.94:0/689772205 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557cb8956b50 con 0x557cb8a08c00 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f7246a00640 10 client.25349 handle_command_reply: tid=0 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 client.25349 shutdown 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e65b0 msgr2=0x557cb89f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e65b0 0x557cb89f5ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72580ed3f0 tx=0x7f7258031490 comp rx=0 tx=0).stop 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 shutdown_connections 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89f85d0 0x557cb89f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a08c00 0x557cb8a0b040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e65b0 0x557cb89f5ae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 --2- 172.21.15.94:0/689772205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f7246a00640 0 client.25349 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 >> 172.21.15.94:0/689772205 conn(0x557cb89adf80 msgr2=0x557cb89e7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 shutdown_connections 2024-06-03T01:11:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.318+0000 7f726ef99740 1 -- 172.21.15.94:0/689772205 wait complete. 2024-06-03T01:11:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.319+0000 7f726ef99740 20 client.25349 trim_cache size 0 max 16384 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 Processor -- start 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 -- start start 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89f85d0 0x557cb89e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72641aca60 con 0x557cb89e65b0 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72641ae5a0 con 0x557cb89f85d0 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.335+0000 7f726ef99740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72641ad130 con 0x557cb89e5040 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44872/0 (socket says 172.21.15.94:44872) 2024-06-03T01:11:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 -- 172.21.15.94:0/3993739913 learned_addr learned my addr 172.21.15.94:0/3993739913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7269600640 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f726a000640 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89f85d0 0x557cb89e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 -- 172.21.15.94:0/3993739913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 msgr2=0x557cb89f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 -- 172.21.15.94:0/3993739913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89f85d0 msgr2=0x557cb89e14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89f85d0 0x557cb89e14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 -- 172.21.15.94:0/3993739913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb8a0b6c0 con 0x557cb89e65b0 2024-06-03T01:11:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7269600640 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7268c00640 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e0f90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f725803b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1225b31689bc5c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.336+0000 7f7251c00640 1 -- 172.21.15.94:0/3993739913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258049070 con 0x557cb89e65b0 2024-06-03T01:11:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f7251c00640 1 -- 172.21.15.94:0/3993739913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72580c05a0 con 0x557cb89e65b0 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f7251c00640 1 -- 172.21.15.94:0/3993739913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72580bbc00 con 0x557cb89e65b0 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 -- 172.21.15.94:0/3993739913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 msgr2=0x557cb89e0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e0f90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7258000c00 tx=0x7f725803b720 comp rx=0 tx=0).stop 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 -- 172.21.15.94:0/3993739913 shutdown_connections 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89f85d0 0x557cb89e14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89e0f90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 --2- 172.21.15.94:0/3993739913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89e5040 0x557cb89f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 -- 172.21.15.94:0/3993739913 >> 172.21.15.94:0/3993739913 conn(0x557cb89adf80 msgr2=0x557cb89f6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 -- 172.21.15.94:0/3993739913 shutdown_connections 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.337+0000 7f726ef99740 1 -- 172.21.15.94:0/3993739913 wait complete. 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.338+0000 7f726ef99740 1 -- start start 2024-06-03T01:11:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.338+0000 7f726ef99740 1 Processor -- start 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726ef99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb8a00d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726ef99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726ef99740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726ef99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72641aded0 con 0x557cb89e65b0 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726ef99740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72641ad800 con 0x557cb89e5040 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726ef99740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72641aaa40 con 0x557cb89f85d0 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726ef99740 10 client.0 _ll_register_callbacks cb 0x7f7258009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44874/0 (socket says 172.21.15.94:44874) 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 -- 172.21.15.94:0/2988114281 learned_addr learned my addr 172.21.15.94:0/2988114281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7269600640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb8a00d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726a000640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 -- 172.21.15.94:0/2988114281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 msgr2=0x557cb89fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 -- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 msgr2=0x557cb8a00d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb8a00d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7268c00640 1 -- 172.21.15.94:0/2988114281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb89fcc80 con 0x557cb89e65b0 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f7269600640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb8a00d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.339+0000 7f726a000640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f7268c00640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89fc100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7258006010 tx=0x7f7258005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1119d9c92d6a00 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f7246a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f7246a00640 1 -- 172.21.15.94:0/2988114281 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258049030 con 0x557cb89e65b0 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f7246a00640 1 -- 172.21.15.94:0/2988114281 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f72580dc270 con 0x557cb89e65b0 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72580c05a0 con 0x557cb89e65b0 2024-06-03T01:11:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f7246a00640 1 -- 172.21.15.94:0/2988114281 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72580bbde0 con 0x557cb89e65b0 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f726ef99740 10 client.15234 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cb89fcc80 con 0x557cb89e65b0 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f7246a00640 1 -- 172.21.15.94:0/2988114281 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72580c05a0 con 0x557cb89e65b0 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f7246a00640 1 -- 172.21.15.94:0/2988114281 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72580ed700 con 0x557cb89e65b0 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f726ef99740 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f726ef99740 10 client.15234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.340+0000 7f726ef99740 20 client.15234 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.341+0000 7f726ef99740 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a73a40 0x557cb8a75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.341+0000 7f726ef99740 4 client.15234 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:11:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.341+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cb8950ea0 con 0x557cb8a73a40 2024-06-03T01:11:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.341+0000 7f7269600640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a73a40 0x557cb8a75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.341+0000 7f7269600640 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a73a40 0x557cb8a75e60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0b2752505b00f4 server_cookie=a21705b6a296bdb5 in_seq=0 out_seq=0 2024-06-03T01:11:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.341+0000 7f7246a00640 10 client.15234 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.611+0000 7f7246a00640 1 -- 172.21.15.94:0/2988114281 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557cb8950ea0 con 0x557cb8a73a40 2024-06-03T01:11:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.611+0000 7f7246a00640 10 client.15234 handle_command_reply: tid=0 2024-06-03T01:11:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.611+0000 7f726ef99740 1 client.15234 shutdown 2024-06-03T01:11:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.611+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 msgr2=0x557cb89fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.611+0000 7f726ef99740 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89fc100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7258006010 tx=0x7f7258005de0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 shutdown_connections 2024-06-03T01:11:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cb89f85d0 0x557cb89fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cb89e65b0 0x557cb89fc100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cb8a73a40 0x557cb8a75e60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 --2- 172.21.15.94:0/2988114281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cb89e5040 0x557cb8a00d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 >> 172.21.15.94:0/2988114281 conn(0x557cb89adf80 msgr2=0x557cb89f7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f7246a00640 0 client.15234 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 shutdown_connections 2024-06-03T01:11:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.612+0000 7f726ef99740 1 -- 172.21.15.94:0/2988114281 wait complete. 2024-06-03T01:11:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.613+0000 7f726ef99740 20 client.15234 trim_cache size 0 max 16384 2024-06-03T01:11:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:11:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:11:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:11:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f724c0a2f40 msgr2=0x7f724c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f724c0a2f40 0x7f724c0a5400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x557cb89e7d10 tx=0x7f7254005f90 comp rx=0 tx=0).stop 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7264108ee0 msgr2=0x7f726419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7264108ee0 0x7f726419aa20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f725802a910 tx=0x7f7258003b20 comp rx=0 tx=0).stop 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 shutdown_connections 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f724c0a2f40 0x7f724c0a5400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726419eda0 0x7f726419f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7264108ee0 0x7f726419aa20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 --2- 172.21.15.94:0/1271185459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7264103e10 0x7f726419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 >> 172.21.15.94:0/1271185459 conn(0x7f72640ff6d0 msgr2=0x7f72641019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 shutdown_connections 2024-06-03T01:11:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:03.614+0000 7f726c000640 1 -- 172.21.15.94:0/1271185459 wait complete. 2024-06-03T01:11:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:04 smithi107 ceph-mon[36622]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-03T01:11:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:04 smithi094 ceph-mon[32119]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-03T01:11:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:04 smithi179 ceph-mon[37041]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-03T01:11:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:06 smithi179 ceph-mon[37041]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-03T01:11:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:06 smithi107 ceph-mon[36622]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-03T01:11:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:06 smithi094 ceph-mon[32119]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-03T01:11:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:08 smithi107 ceph-mon[36622]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-03T01:11:08.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:08 smithi094 ceph-mon[32119]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-03T01:11:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:08 smithi179 ceph-mon[37041]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-03T01:11:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:10 smithi094 ceph-mon[32119]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-06-03T01:11:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:10 smithi107 ceph-mon[36622]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-06-03T01:11:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:10 smithi179 ceph-mon[37041]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-06-03T01:11:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:12 smithi107 ceph-mon[36622]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-03T01:11:12.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:12 smithi094 ceph-mon[32119]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-03T01:11:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:12 smithi179 ceph-mon[37041]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-03T01:11:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:14 smithi107 ceph-mon[36622]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-06-03T01:11:14.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:14 smithi094 ceph-mon[32119]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-06-03T01:11:14.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:14 smithi179 ceph-mon[37041]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-06-03T01:11:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 -- 172.21.15.94:0/207689404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1017b0 msgr2=0x7ff93c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 --2- 172.21.15.94:0/207689404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1017b0 0x7ff93c101c30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff928009890 tx=0x7ff92802a360 comp rx=0 tx=0).stop 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 -- 172.21.15.94:0/207689404 shutdown_connections 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 --2- 172.21.15.94:0/207689404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff93c102170 0x7ff93c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 --2- 172.21.15.94:0/207689404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1017b0 0x7ff93c101c30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 --2- 172.21.15.94:0/207689404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1077c0 0x7ff93c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 -- 172.21.15.94:0/207689404 >> 172.21.15.94:0/207689404 conn(0x7ff93c0fd320 msgr2=0x7ff93c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 -- 172.21.15.94:0/207689404 shutdown_connections 2024-06-03T01:11:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.771+0000 7ff940c00640 1 -- 172.21.15.94:0/207689404 wait complete. 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 Processor -- start 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 -- start start 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1017b0 0x7ff93c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff93c102170 0x7ff93c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1077c0 0x7ff93c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c111bd0 con 0x7ff93c1017b0 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c111a50 con 0x7ff93c102170 2024-06-03T01:11:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.773+0000 7ff940c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c111d50 con 0x7ff93c1077c0 2024-06-03T01:11:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1017b0 0x7ff93c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1017b0 0x7ff93c198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54120/0 (socket says 172.21.15.94:54120) 2024-06-03T01:11:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 -- 172.21.15.94:0/3108353551 learned_addr learned my addr 172.21.15.94:0/3108353551 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93aa00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff93c102170 0x7ff93c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93be00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1077c0 0x7ff93c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 -- 172.21.15.94:0/3108353551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1077c0 msgr2=0x7ff93c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1077c0 0x7ff93c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 -- 172.21.15.94:0/3108353551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff93c102170 msgr2=0x7ff93c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff93c102170 0x7ff93c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93b400640 1 -- 172.21.15.94:0/3108353551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c19d120 con 0x7ff93c1017b0 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93be00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1077c0 0x7ff93c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.774+0000 7ff93aa00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff93c102170 0x7ff93c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.775+0000 7ff93b400640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1017b0 0x7ff93c198130 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00e9c0 tx=0x7ff92c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.775+0000 7ff933e00640 1 -- 172.21.15.94:0/3108353551 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00ccc0 con 0x7ff93c1017b0 2024-06-03T01:11:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.775+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c19d410 con 0x7ff93c1017b0 2024-06-03T01:11:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.775+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c1a4ca0 con 0x7ff93c1017b0 2024-06-03T01:11:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.775+0000 7ff933e00640 1 -- 172.21.15.94:0/3108353551 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff92c005500 con 0x7ff93c1017b0 2024-06-03T01:11:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.775+0000 7ff933e00640 1 -- 172.21.15.94:0/3108353551 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c010430 con 0x7ff93c1017b0 2024-06-03T01:11:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 Processor -- start 2024-06-03T01:11:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff933e00640 1 -- 172.21.15.94:0/3108353551 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7ff92c0105d0 con 0x7ff93c1017b0 2024-06-03T01:11:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 -- start start 2024-06-03T01:11:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c8ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8d200 0x55f7f9c8d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c9a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9bf1420 con 0x55f7f9c8e770 2024-06-03T01:11:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9b98c50 con 0x55f7f9c97f80 2024-06-03T01:11:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff944f68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9bed3e0 con 0x55f7f9c8d200 2024-06-03T01:11:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.779+0000 7ff93be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c8ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c8ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54124/0 (socket says 172.21.15.94:54124) 2024-06-03T01:11:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 -- 172.21.15.94:0/1119513251 learned_addr learned my addr 172.21.15.94:0/1119513251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93b400640 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8d200 0x55f7f9c8d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93aa00640 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c9a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 -- 172.21.15.94:0/1119513251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8d200 msgr2=0x55f7f9c8d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8d200 0x55f7f9c8d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 -- 172.21.15.94:0/1119513251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 msgr2=0x55f7f9c9a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c9a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 -- 172.21.15.94:0/1119513251 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f9905210 con 0x55f7f9c8e770 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff93be00640 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c8ccc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff934009fd0 tx=0x7ff9340108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a50f078e6aeb56 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.780+0000 7ff933e00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff9200a2e90 0x7ff9200a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.781+0000 7ff927e00640 1 -- 172.21.15.94:0/1119513251 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934011760 con 0x55f7f9c8e770 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.781+0000 7ff927e00640 1 -- 172.21.15.94:0/1119513251 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff934011de0 con 0x55f7f9c8e770 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.781+0000 7ff927e00640 1 -- 172.21.15.94:0/1119513251 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93400e3c0 con 0x55f7f9c8e770 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.781+0000 7ff933e00640 1 -- 172.21.15.94:0/3108353551 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7ff92c0cfc40 con 0x7ff93c1017b0 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.781+0000 7ff93b400640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff9200a2e90 0x7ff9200a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 -- 172.21.15.94:0/1119513251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 msgr2=0x55f7f9c8ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c8ccc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff934009fd0 tx=0x7ff9340108e0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff93b400640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff9200a2e90 0x7ff9200a5350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0cceb0 tx=0x7ff92c0d36d0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 -- 172.21.15.94:0/1119513251 shutdown_connections 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c9a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8d200 0x55f7f9c8d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 --2- 172.21.15.94:0/1119513251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c8ccc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 -- 172.21.15.94:0/1119513251 >> 172.21.15.94:0/1119513251 conn(0x55f7f9c55f80 msgr2=0x55f7f9c897f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 -- 172.21.15.94:0/1119513251 shutdown_connections 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 -- 172.21.15.94:0/1119513251 wait complete. 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.782+0000 7ff944f68740 1 -- start start 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 1 Processor -- start 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9c94f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c95450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9c95990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9c9ad50 con 0x55f7f9c8e770 2024-06-03T01:11:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9bac0b0 con 0x55f7f9c8d200 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9c9aed0 con 0x55f7f9c97f80 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff944f68740 10 client.0 _ll_register_callbacks cb 0x55f7f9b97f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff93be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9c94f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff93be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9c94f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:38234/0 (socket says 172.21.15.94:38234) 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff93be00640 1 -- 172.21.15.94:0/3031829356 learned_addr learned my addr 172.21.15.94:0/3031829356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff93aa00640 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c95450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.783+0000 7ff93be00640 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9c95990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff93aa00640 1 -- 172.21.15.94:0/3031829356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 msgr2=0x55f7f9c95990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff93aa00640 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9c95990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff93aa00640 1 -- 172.21.15.94:0/3031829356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 msgr2=0x55f7f9c94f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff93aa00640 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9c94f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff93aa00640 1 -- 172.21.15.94:0/3031829356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f99d1620 con 0x55f7f9c8e770 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff93aa00640 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c95450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff92802a880 tx=0x7ff9280045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8ec986bc68d398 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff924c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff924c00640 1 -- 172.21.15.94:0/3031829356 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928037040 con 0x55f7f9c8e770 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff924c00640 1 -- 172.21.15.94:0/3031829356 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7ff928030990 con 0x55f7f9c8e770 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.784+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff934011de0 con 0x55f7f9c8e770 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff924c00640 1 -- 172.21.15.94:0/3031829356 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff92804c050 con 0x55f7f9c8e770 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 10 client.15246 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7f9ca7030 con 0x55f7f9c8e770 2024-06-03T01:11:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff924c00640 1 -- 172.21.15.94:0/3031829356 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff934011de0 con 0x55f7f9c8e770 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff924c00640 1 -- 172.21.15.94:0/3031829356 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff92803d6c0 con 0x55f7f9c8e770 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 10 client.15246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 20 client.15246 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9ca8540 0x55f7f9caa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 4 client.15246 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.785+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7f9bfeb50 con 0x55f7f9ca8540 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.786+0000 7ff93be00640 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9ca8540 0x55f7f9caa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.786+0000 7ff93be00640 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9ca8540 0x55f7f9caa990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679e9f489f5dd6fd server_cookie=6028852c07ddad99 in_seq=0 out_seq=0 2024-06-03T01:11:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.786+0000 7ff924c00640 10 client.15246 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:16.794 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:16 smithi107 ceph-mon[36622]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T01:11:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff924c00640 1 -- 172.21.15.94:0/3031829356 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f7f9bfeb50 con 0x55f7f9ca8540 2024-06-03T01:11:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff924c00640 10 client.15246 handle_command_reply: tid=0 2024-06-03T01:11:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 client.15246 shutdown 2024-06-03T01:11:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 msgr2=0x55f7f9c95450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c95450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff92802a880 tx=0x7ff9280045f0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 shutdown_connections 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9c95990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9c95450 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9ca8540 0x55f7f9caa990 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 --2- 172.21.15.94:0/3031829356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9c94f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 >> 172.21.15.94:0/3031829356 conn(0x55f7f9c55f80 msgr2=0x55f7f9c8f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.787+0000 7ff924c00640 0 client.15246 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.788+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 shutdown_connections 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.788+0000 7ff944f68740 1 -- 172.21.15.94:0/3031829356 wait complete. 2024-06-03T01:11:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.788+0000 7ff944f68740 20 client.15246 trim_cache size 0 max 16384 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.819+0000 7ff944f68740 1 Processor -- start 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff944f68740 1 -- start start 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff944f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8d200 0x55f7f9ca0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff944f68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8e770 0x55f7f9ca0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff944f68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff944f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c00fd50 con 0x55f7f9c8d200 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff944f68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c011890 con 0x55f7f9c97f80 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff944f68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c010420 con 0x55f7f9c8e770 2024-06-03T01:11:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff93be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8d200 0x55f7f9ca0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff93be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8d200 0x55f7f9ca0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54134/0 (socket says 172.21.15.94:54134) 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff93be00640 1 -- 172.21.15.94:0/244650210 learned_addr learned my addr 172.21.15.94:0/244650210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff93aa00640 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8e770 0x55f7f9ca0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.820+0000 7ff93be00640 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff93be00640 1 -- 172.21.15.94:0/244650210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8e770 msgr2=0x55f7f9ca0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff93be00640 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8e770 0x55f7f9ca0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff93be00640 1 -- 172.21.15.94:0/244650210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 msgr2=0x55f7f9c890f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff93be00640 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c890f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff93be00640 1 -- 172.21.15.94:0/244650210 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f9c8df30 con 0x55f7f9c8d200 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff93be00640 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8d200 0x55f7f9ca0540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff93401baf0 tx=0x7ff934009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e048ff762ffa143 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff927400640 1 -- 172.21.15.94:0/244650210 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93401b430 con 0x55f7f9c8d200 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff927400640 1 -- 172.21.15.94:0/244650210 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93401b5d0 con 0x55f7f9c8d200 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.821+0000 7ff927400640 1 -- 172.21.15.94:0/244650210 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934011ca0 con 0x55f7f9c8d200 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 -- 172.21.15.94:0/244650210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8d200 msgr2=0x55f7f9ca0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8d200 0x55f7f9ca0540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff93401baf0 tx=0x7ff934009500 comp rx=0 tx=0).stop 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 -- 172.21.15.94:0/244650210 shutdown_connections 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c97f80 0x55f7f9c890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c8e770 0x55f7f9ca0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 --2- 172.21.15.94:0/244650210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8d200 0x55f7f9ca0540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 -- 172.21.15.94:0/244650210 >> 172.21.15.94:0/244650210 conn(0x55f7f9c55f80 msgr2=0x55f7f9c8f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 -- 172.21.15.94:0/244650210 shutdown_connections 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.822+0000 7ff944f68740 1 -- 172.21.15.94:0/244650210 wait complete. 2024-06-03T01:11:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 -- start start 2024-06-03T01:11:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 Processor -- start 2024-06-03T01:11:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9d0b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9d0b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9d0bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c0111c0 con 0x55f7f9c8e770 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c010af0 con 0x55f7f9c8d200 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c00dd30 con 0x55f7f9c97f80 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.823+0000 7ff944f68740 10 client.0 _ll_register_callbacks cb 0x7ff9340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9d0b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9d0b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9d0b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:38252/0 (socket says 172.21.15.94:38252) 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 -- 172.21.15.94:0/681619114 learned_addr learned my addr 172.21.15.94:0/681619114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93aa00640 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9d0b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54138/0 (socket says 172.21.15.94:54138) 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9d0bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 -- 172.21.15.94:0/681619114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 msgr2=0x55f7f9d0bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9d0bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 -- 172.21.15.94:0/681619114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 msgr2=0x55f7f9d0b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9d0b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.824+0000 7ff93be00640 1 -- 172.21.15.94:0/681619114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f9d163d0 con 0x55f7f9c8d200 2024-06-03T01:11:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff93be00640 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9d0b1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff934009e10 tx=0x7ff934011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5aa584f2e84fda server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff924c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff924c00640 1 -- 172.21.15.94:0/681619114 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93401b780 con 0x55f7f9c8d200 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93401b5d0 con 0x55f7f9c8d200 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff924c00640 1 -- 172.21.15.94:0/681619114 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7ff934023040 con 0x55f7f9c8d200 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff924c00640 1 -- 172.21.15.94:0/681619114 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff934011a20 con 0x55f7f9c8d200 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff944f68740 10 client.25373 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7f9d176e0 con 0x55f7f9c8d200 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.825+0000 7ff924c00640 1 -- 172.21.15.94:0/681619114 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93401b5d0 con 0x55f7f9c8d200 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff924c00640 1 -- 172.21.15.94:0/681619114 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff934029040 con 0x55f7f9c8d200 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff944f68740 10 client.25373 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff944f68740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff944f68740 20 client.25373 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff944f68740 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9d17b00 0x55f7f9d19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff944f68740 4 client.25373 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:11:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7f9bf8ea0 con 0x55f7f9d17b00 2024-06-03T01:11:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.826+0000 7ff93aa00640 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9d17b00 0x55f7f9d19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.827+0000 7ff924c00640 10 client.25373 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.827+0000 7ff93aa00640 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9d17b00 0x55f7f9d19ef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6832d25e221ec4 server_cookie=16e93f3bfac8547f in_seq=0 out_seq=0 2024-06-03T01:11:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.830+0000 7ff924c00640 1 -- 172.21.15.94:0/681619114 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f7f9bf8ea0 con 0x55f7f9d17b00 2024-06-03T01:11:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.830+0000 7ff924c00640 10 client.25373 handle_command_reply: tid=0 2024-06-03T01:11:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.830+0000 7ff944f68740 1 client.25373 shutdown 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.830+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 msgr2=0x55f7f9d0b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.830+0000 7ff944f68740 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9d0b1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff934009e10 tx=0x7ff934011880 comp rx=0 tx=0).stop 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 shutdown_connections 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f7f9c97f80 0x55f7f9d0bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f7f9c8e770 0x55f7f9d0b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f7f9d17b00 0x55f7f9d19ef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 --2- 172.21.15.94:0/681619114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7f9c8d200 0x55f7f9d0b1f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 >> 172.21.15.94:0/681619114 conn(0x55f7f9c55f80 msgr2=0x55f7f9c94eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff924c00640 0 client.25373 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 shutdown_connections 2024-06-03T01:11:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 1 -- 172.21.15.94:0/681619114 wait complete. 2024-06-03T01:11:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.831+0000 7ff944f68740 20 client.25373 trim_cache size 0 max 16384 2024-06-03T01:11:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:11:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:11:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff9200a2e90 msgr2=0x7ff9200a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff9200a2e90 0x7ff9200a5350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0cceb0 tx=0x7ff92c0d36d0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1017b0 msgr2=0x7ff93c198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1017b0 0x7ff93c198130 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00e9c0 tx=0x7ff92c00ee80 comp rx=0 tx=0).stop 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 shutdown_connections 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff9200a2e90 0x7ff9200a5350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff93c1077c0 0x7ff93c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff93c102170 0x7ff93c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 --2- 172.21.15.94:0/3108353551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff93c1017b0 0x7ff93c198130 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 >> 172.21.15.94:0/3108353551 conn(0x7ff93c0fd320 msgr2=0x7ff93c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 shutdown_connections 2024-06-03T01:11:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:16.832+0000 7ff940c00640 1 -- 172.21.15.94:0/3108353551 wait complete. 2024-06-03T01:11:16.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:16 smithi094 ceph-mon[32119]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T01:11:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:16 smithi179 ceph-mon[37041]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T01:11:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:18 smithi107 ceph-mon[36622]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T01:11:18.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:18 smithi094 ceph-mon[32119]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T01:11:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:18 smithi179 ceph-mon[37041]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-06-03T01:11:20.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:20 smithi107 ceph-mon[36622]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-06-03T01:11:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:20 smithi094 ceph-mon[32119]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-06-03T01:11:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:20 smithi179 ceph-mon[37041]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-06-03T01:11:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:22 smithi107 ceph-mon[36622]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:22.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:22 smithi094 ceph-mon[32119]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:22.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:22 smithi179 ceph-mon[37041]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:24 smithi107 ceph-mon[36622]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:24.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:24 smithi094 ceph-mon[32119]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:24 smithi179 ceph-mon[37041]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:26 smithi107 ceph-mon[36622]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:26.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:26 smithi094 ceph-mon[32119]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:26 smithi179 ceph-mon[37041]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:28 smithi107 ceph-mon[36622]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:28.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:28 smithi094 ceph-mon[32119]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:28 smithi179 ceph-mon[37041]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 -- 172.21.15.94:0/1457961292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48075ed0 msgr2=0x7f5e480762b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/1457961292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48075ed0 0x7f5e480762b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009930 tx=0x7f5e3c02a440 comp rx=0 tx=0).stop 2024-06-03T01:11:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 -- 172.21.15.94:0/1457961292 shutdown_connections 2024-06-03T01:11:30.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/1457961292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e48112830 0x7f5e48114c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/1457961292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e480767f0 0x7f5e481122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/1457961292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48075ed0 0x7f5e480762b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 -- 172.21.15.94:0/1457961292 >> 172.21.15.94:0/1457961292 conn(0x7f5e480ff6a0 msgr2=0x7f5e48101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 -- 172.21.15.94:0/1457961292 shutdown_connections 2024-06-03T01:11:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.018+0000 7f5e4fa00640 1 -- 172.21.15.94:0/1457961292 wait complete. 2024-06-03T01:11:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.019+0000 7f5e4fa00640 1 Processor -- start 2024-06-03T01:11:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.019+0000 7f5e4fa00640 1 -- start start 2024-06-03T01:11:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.020+0000 7f5e4fa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e48075ed0 0x7f5e481a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.020+0000 7f5e4fa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e480767f0 0x7f5e481a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.020+0000 7f5e4fa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48112830 0x7f5e4819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.020+0000 7f5e4fa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e481183d0 con 0x7f5e48075ed0 2024-06-03T01:11:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.020+0000 7f5e4fa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48118250 con 0x7f5e48112830 2024-06-03T01:11:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.020+0000 7f5e4fa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48118550 con 0x7f5e480767f0 2024-06-03T01:11:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e47e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e480767f0 0x7f5e481a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e48075ed0 0x7f5e481a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e48075ed0 0x7f5e481a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56288/0 (socket says 172.21.15.94:56288) 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4d000640 1 -- 172.21.15.94:0/453274553 learned_addr learned my addr 172.21.15.94:0/453274553 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4da00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48112830 0x7f5e4819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4da00640 1 -- 172.21.15.94:0/453274553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e480767f0 msgr2=0x7f5e481a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4da00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e480767f0 0x7f5e481a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4da00640 1 -- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e48075ed0 msgr2=0x7f5e481a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4da00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e48075ed0 0x7f5e481a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e4da00640 1 -- 172.21.15.94:0/453274553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4819efd0 con 0x7f5e48112830 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.021+0000 7f5e47e00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e480767f0 0x7f5e481a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.022+0000 7f5e4da00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48112830 0x7f5e4819e710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5e40005fd0 tx=0x7f5e4000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.022+0000 7f5e45600640 1 -- 172.21.15.94:0/453274553 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000cdf0 con 0x7f5e48112830 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.022+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4819f2c0 con 0x7f5e48112830 2024-06-03T01:11:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.023+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e481ab580 con 0x7f5e48112830 2024-06-03T01:11:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.023+0000 7f5e45600640 1 -- 172.21.15.94:0/453274553 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e40005020 con 0x7f5e48112830 2024-06-03T01:11:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.023+0000 7f5e45600640 1 -- 172.21.15.94:0/453274553 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e400106d0 con 0x7f5e48112830 2024-06-03T01:11:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.026+0000 7f5e45600640 1 -- 172.21.15.94:0/453274553 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f5e40004a40 con 0x7f5e48112830 2024-06-03T01:11:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.027+0000 7f5e45600640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5e300a2f40 0x7f5e300a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.028+0000 7f5e45600640 1 -- 172.21.15.94:0/453274553 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f5e400d00b0 con 0x7f5e48112830 2024-06-03T01:11:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.028+0000 7f5e4d000640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5e300a2f40 0x7f5e300a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.029+0000 7f5e4d000640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5e300a2f40 0x7f5e300a5400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0096c0 tx=0x7f5e3c005b50 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.029+0000 7f5e529dd740 1 Processor -- start 2024-06-03T01:11:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e529dd740 1 -- start start 2024-06-03T01:11:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e111ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111b040 0x5597e111b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e1125dc0 0x5597e11281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e529dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597e107f420 con 0x5597e111b040 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e529dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597e1026c50 con 0x5597e111c5b0 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e529dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5597e107b3e0 con 0x5597e1125dc0 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e111ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e111ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57682/0 (socket says 172.21.15.94:57682) 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.030+0000 7f5e4d000640 1 -- 172.21.15.94:0/176738393 learned_addr learned my addr 172.21.15.94:0/176738393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e47e00640 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111b040 0x5597e111b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e4da00640 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e1125dc0 0x5597e11281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e4d000640 1 -- 172.21.15.94:0/176738393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e1125dc0 msgr2=0x5597e11281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e4d000640 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e1125dc0 0x5597e11281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e4d000640 1 -- 172.21.15.94:0/176738393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111b040 msgr2=0x5597e111b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e4d000640 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111b040 0x5597e111b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e4d000640 1 -- 172.21.15.94:0/176738393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e0d93210 con 0x5597e111c5b0 2024-06-03T01:11:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.031+0000 7f5e47e00640 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111b040 0x5597e111b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.032+0000 7f5e4d000640 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e111ab00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0044a0 tx=0x7f5e3c03d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad8495c5011a28e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.032+0000 7f5e3a000640 1 -- 172.21.15.94:0/176738393 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c035040 con 0x5597e111c5b0 2024-06-03T01:11:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.032+0000 7f5e3a000640 1 -- 172.21.15.94:0/176738393 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3c04e070 con 0x5597e111c5b0 2024-06-03T01:11:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.032+0000 7f5e3a000640 1 -- 172.21.15.94:0/176738393 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c032d90 con 0x5597e111c5b0 2024-06-03T01:11:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.033+0000 7f5e529dd740 1 -- 172.21.15.94:0/176738393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 msgr2=0x5597e111ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.033+0000 7f5e529dd740 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e111ab00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0044a0 tx=0x7f5e3c03d000 comp rx=0 tx=0).stop 2024-06-03T01:11:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.034+0000 7f5e529dd740 1 -- 172.21.15.94:0/176738393 shutdown_connections 2024-06-03T01:11:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.034+0000 7f5e529dd740 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e1125dc0 0x5597e11281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.034+0000 7f5e529dd740 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111b040 0x5597e111b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.034+0000 7f5e529dd740 1 --2- 172.21.15.94:0/176738393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e111ab00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.034+0000 7f5e529dd740 1 -- 172.21.15.94:0/176738393 >> 172.21.15.94:0/176738393 conn(0x5597e10e3f80 msgr2=0x5597e1117630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.034+0000 7f5e529dd740 1 -- 172.21.15.94:0/176738393 shutdown_connections 2024-06-03T01:11:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.034+0000 7f5e529dd740 1 -- 172.21.15.94:0/176738393 wait complete. 2024-06-03T01:11:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.035+0000 7f5e529dd740 1 -- start start 2024-06-03T01:11:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.035+0000 7f5e529dd740 1 Processor -- start 2024-06-03T01:11:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1122d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e11232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e1123810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e529dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5597e1139b40 con 0x5597e1125dc0 2024-06-03T01:11:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e529dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5597e103a0b0 con 0x5597e111c5b0 2024-06-03T01:11:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e529dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5597e1139cc0 con 0x5597e111b040 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e529dd740 10 client.0 _ll_register_callbacks cb 0x5597e1025f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1122d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1122d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:60728/0 (socket says 172.21.15.94:60728) 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.036+0000 7f5e4d000640 1 -- 172.21.15.94:0/1826454243 learned_addr learned my addr 172.21.15.94:0/1826454243 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e47e00640 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e11232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e4da00640 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e1123810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e4d000640 1 -- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 msgr2=0x5597e11232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e4d000640 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e11232d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e4d000640 1 -- 172.21.15.94:0/1826454243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 msgr2=0x5597e1123810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e4d000640 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e1123810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e4d000640 1 -- 172.21.15.94:0/1826454243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e0e5f620 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e4d000640 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1122d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c006010 tx=0x7f5e3c0056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7018651a62cb8463 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.037+0000 7f5e2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.038+0000 7f5e2ea00640 1 -- 172.21.15.94:0/1826454243 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c035030 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.038+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e3c04e070 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.038+0000 7f5e2ea00640 1 -- 172.21.15.94:0/1826454243 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f5e3c032cd0 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.038+0000 7f5e2ea00640 1 -- 172.21.15.94:0/1826454243 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e3c065050 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.038+0000 7f5e529dd740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.038+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597e1139b40 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.038+0000 7f5e2ea00640 1 -- 172.21.15.94:0/1826454243 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3c04e070 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e2ea00640 1 -- 172.21.15.94:0/1826454243 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5e3c003820 con 0x5597e111b040 2024-06-03T01:11:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e529dd740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e529dd740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e529dd740 20 client.25399 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e529dd740 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e1147020 0x5597e1149410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e529dd740 4 client.25399 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:11:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597e108cb50 con 0x5597e1147020 2024-06-03T01:11:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.039+0000 7f5e47e00640 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e1147020 0x5597e1149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.040+0000 7f5e2ea00640 10 client.25399 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.040+0000 7f5e47e00640 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e1147020 0x5597e1149410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313056907445ad06 server_cookie=57c2901bedfd5cb6 in_seq=0 out_seq=0 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e2ea00640 1 -- 172.21.15.94:0/1826454243 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597e108cb50 con 0x5597e1147020 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e2ea00640 10 client.25399 handle_command_reply: tid=0 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 client.25399 shutdown 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 msgr2=0x5597e1122d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1122d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c006010 tx=0x7f5e3c0056b0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 shutdown_connections 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e1123810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e1147020 0x5597e1149410 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111c5b0 0x5597e11232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 --2- 172.21.15.94:0/1826454243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1122d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e2ea00640 0 client.25399 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.041+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 >> 172.21.15.94:0/1826454243 conn(0x5597e10e3f80 msgr2=0x5597e111d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.042+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 shutdown_connections 2024-06-03T01:11:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.042+0000 7f5e529dd740 1 -- 172.21.15.94:0/1826454243 wait complete. 2024-06-03T01:11:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.042+0000 7f5e529dd740 20 client.25399 trim_cache size 0 max 16384 2024-06-03T01:11:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.058+0000 7f5e529dd740 1 Processor -- start 2024-06-03T01:11:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e529dd740 1 -- start start 2024-06-03T01:11:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1123630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111c5b0 0x5597e114ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e1125dc0 0x5597e114f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e529dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2800fdf0 con 0x5597e111c5b0 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e529dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28011930 con 0x5597e1125dc0 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e529dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e280104c0 con 0x5597e111b040 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1123630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1123630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:60740/0 (socket says 172.21.15.94:60740) 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e4d000640 1 -- 172.21.15.94:0/955290863 learned_addr learned my addr 172.21.15.94:0/955290863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.059+0000 7f5e4da00640 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e1125dc0 0x5597e114f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e47e00640 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111c5b0 0x5597e114ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e4d000640 1 -- 172.21.15.94:0/955290863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e1125dc0 msgr2=0x5597e114f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e4d000640 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e1125dc0 0x5597e114f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e4d000640 1 -- 172.21.15.94:0/955290863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111c5b0 msgr2=0x5597e114ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e4d000640 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111c5b0 0x5597e114ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e4d000640 1 -- 172.21.15.94:0/955290863 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e1112ff0 con 0x5597e111b040 2024-06-03T01:11:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e47e00640 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111c5b0 0x5597e114ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:11:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e4d000640 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1123630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009900 tx=0x7f5e3c0097c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc63077c2178b483 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e39600640 1 -- 172.21.15.94:0/955290863 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c035070 con 0x5597e111b040 2024-06-03T01:11:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e39600640 1 -- 172.21.15.94:0/955290863 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3c04e040 con 0x5597e111b040 2024-06-03T01:11:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.060+0000 7f5e39600640 1 -- 172.21.15.94:0/955290863 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c040040 con 0x5597e111b040 2024-06-03T01:11:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 -- 172.21.15.94:0/955290863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 msgr2=0x5597e1123630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1123630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009900 tx=0x7f5e3c0097c0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 -- 172.21.15.94:0/955290863 shutdown_connections 2024-06-03T01:11:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e1125dc0 0x5597e114f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e111c5b0 0x5597e114ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 --2- 172.21.15.94:0/955290863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111b040 0x5597e1123630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 -- 172.21.15.94:0/955290863 >> 172.21.15.94:0/955290863 conn(0x5597e10e3f80 msgr2=0x5597e111d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 -- 172.21.15.94:0/955290863 shutdown_connections 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.061+0000 7f5e529dd740 1 -- 172.21.15.94:0/955290863 wait complete. 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.062+0000 7f5e529dd740 1 -- start start 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.062+0000 7f5e529dd740 1 Processor -- start 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.062+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111b040 0x5597e113f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111c5b0 0x5597e113a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e529dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e113ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e529dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28011260 con 0x5597e1125dc0 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e529dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e28010b90 con 0x5597e111b040 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e529dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2800ddd0 con 0x5597e111c5b0 2024-06-03T01:11:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e529dd740 10 client.0 _ll_register_callbacks cb 0x7f5e3400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e47e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111c5b0 0x5597e113a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111b040 0x5597e113f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111b040 0x5597e113f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57702/0 (socket says 172.21.15.94:57702) 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4d000640 1 -- 172.21.15.94:0/3230281753 learned_addr learned my addr 172.21.15.94:0/3230281753 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4da00640 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e113ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4d000640 1 -- 172.21.15.94:0/3230281753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111c5b0 msgr2=0x5597e113a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4d000640 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111c5b0 0x5597e113a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4d000640 1 -- 172.21.15.94:0/3230281753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 msgr2=0x5597e113ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.063+0000 7f5e4d000640 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e113ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.064+0000 7f5e4d000640 1 -- 172.21.15.94:0/3230281753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e113b420 con 0x5597e111b040 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.064+0000 7f5e4d000640 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111b040 0x5597e113f530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c03f040 tx=0x7f5e3c0041b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de11bf3545fd1a3e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.064+0000 7f5e2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:11:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.064+0000 7f5e2ea00640 1 -- 172.21.15.94:0/3230281753 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c040040 con 0x5597e111b040 2024-06-03T01:11:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.064+0000 7f5e2ea00640 1 -- 172.21.15.94:0/3230281753 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f5e3c032b40 con 0x5597e111b040 2024-06-03T01:11:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.064+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e3c04e040 con 0x5597e111b040 2024-06-03T01:11:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.064+0000 7f5e2ea00640 1 -- 172.21.15.94:0/3230281753 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e3c076050 con 0x5597e111b040 2024-06-03T01:11:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 10 client.25391 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597e11b19c0 con 0x5597e111b040 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e2ea00640 1 -- 172.21.15.94:0/3230281753 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3c04e040 con 0x5597e111b040 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e2ea00640 1 -- 172.21.15.94:0/3230281753 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5e3c071030 con 0x5597e111b040 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 10 client.25391 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 10 client.25391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 20 client.25391 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e11b1fc0 0x5597e11b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 4 client.25391 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.065+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597e1086ea0 con 0x5597e11b1fc0 2024-06-03T01:11:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.066+0000 7f5e47e00640 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e11b1fc0 0x5597e11b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.066+0000 7f5e2ea00640 10 client.25391 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.066+0000 7f5e47e00640 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e11b1fc0 0x5597e11b43e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38d00518def9804 server_cookie=64822c2e2a3dfea3 in_seq=0 out_seq=0 2024-06-03T01:11:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.069+0000 7f5e2ea00640 1 -- 172.21.15.94:0/3230281753 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5597e1086ea0 con 0x5597e11b1fc0 2024-06-03T01:11:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.069+0000 7f5e2ea00640 10 client.25391 handle_command_reply: tid=0 2024-06-03T01:11:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 client.25391 shutdown 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111b040 msgr2=0x5597e113f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111b040 0x5597e113f530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c03f040 tx=0x7f5e3c0041b0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 shutdown_connections 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5597e1125dc0 0x5597e113ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597e111c5b0 0x5597e113a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5597e11b1fc0 0x5597e11b43e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 --2- 172.21.15.94:0/3230281753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5597e111b040 0x5597e113f530 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 >> 172.21.15.94:0/3230281753 conn(0x5597e10e3f80 msgr2=0x5597e1115f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.070+0000 7f5e2ea00640 0 client.25391 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.071+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 shutdown_connections 2024-06-03T01:11:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.071+0000 7f5e529dd740 1 -- 172.21.15.94:0/3230281753 wait complete. 2024-06-03T01:11:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.071+0000 7f5e529dd740 20 client.25391 trim_cache size 0 max 16384 2024-06-03T01:11:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:11:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:11:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5e300a2f40 msgr2=0x7f5e300a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5e300a2f40 0x7f5e300a5400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0096c0 tx=0x7f5e3c005b50 comp rx=0 tx=0).stop 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48112830 msgr2=0x7f5e4819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48112830 0x7f5e4819e710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5e40005fd0 tx=0x7f5e4000ef10 comp rx=0 tx=0).stop 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 shutdown_connections 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5e300a2f40 0x7f5e300a5400 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e48112830 0x7f5e4819e710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e480767f0 0x7f5e481a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 --2- 172.21.15.94:0/453274553 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e48075ed0 0x7f5e481a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 >> 172.21.15.94:0/453274553 conn(0x7f5e480ff6a0 msgr2=0x7f5e48114340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 shutdown_connections 2024-06-03T01:11:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:30.072+0000 7f5e4fa00640 1 -- 172.21.15.94:0/453274553 wait complete. 2024-06-03T01:11:30.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:30 smithi107 ceph-mon[36622]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:30.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:30 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:30.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:30 smithi094 ceph-mon[32119]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:30.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:30 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:30 smithi179 ceph-mon[37041]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:30 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:11:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:31 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:11:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:31 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:11:31.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:31 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:11:31.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:31 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:11:31.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:31 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:11:31.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:31 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:11:31.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:31 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:11:31.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:31 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:11:31.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:31 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:11:32.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:32 smithi107 ceph-mon[36622]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:32.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:32 smithi094 ceph-mon[32119]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:32.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:32 smithi179 ceph-mon[37041]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:34.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:34 smithi107 ceph-mon[36622]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-06-03T01:11:34.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:34 smithi094 ceph-mon[32119]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-06-03T01:11:34.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:34 smithi179 ceph-mon[37041]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 12 op/s 2024-06-03T01:11:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:36 smithi107 ceph-mon[36622]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:36 smithi094 ceph-mon[32119]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:36 smithi179 ceph-mon[37041]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:38 smithi107 ceph-mon[36622]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:38.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:38 smithi094 ceph-mon[32119]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:38 smithi179 ceph-mon[37041]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:39.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:39 smithi107 ceph-mon[36622]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:39.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:39 smithi094 ceph-mon[32119]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:39.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:39 smithi179 ceph-mon[37041]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:42.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:41 smithi094 ceph-mon[32119]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:42.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:41 smithi179 ceph-mon[37041]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:42.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:41 smithi107 ceph-mon[36622]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-03T01:11:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.252+0000 7f4a7d000640 1 -- 172.21.15.94:0/2696338115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 msgr2=0x7f4a78109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.252+0000 7f4a7d000640 1 --2- 172.21.15.94:0/2696338115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 0x7f4a78109dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009930 tx=0x7f4a6802a440 comp rx=0 tx=0).stop 2024-06-03T01:11:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.253+0000 7f4a7d000640 1 -- 172.21.15.94:0/2696338115 shutdown_connections 2024-06-03T01:11:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.253+0000 7f4a7d000640 1 --2- 172.21.15.94:0/2696338115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a78104390 0x7f4a781108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.253+0000 7f4a7d000640 1 --2- 172.21.15.94:0/2696338115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781039d0 0x7f4a78103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.253+0000 7f4a7d000640 1 --2- 172.21.15.94:0/2696338115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 0x7f4a78109dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.253+0000 7f4a7d000640 1 -- 172.21.15.94:0/2696338115 >> 172.21.15.94:0/2696338115 conn(0x7f4a780ff480 msgr2=0x7f4a781018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.253+0000 7f4a7d000640 1 -- 172.21.15.94:0/2696338115 shutdown_connections 2024-06-03T01:11:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.253+0000 7f4a7d000640 1 -- 172.21.15.94:0/2696338115 wait complete. 2024-06-03T01:11:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 Processor -- start 2024-06-03T01:11:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 -- start start 2024-06-03T01:11:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781039d0 0x7f4a7819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a78104390 0x7f4a7819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 0x7f4a7819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78113e10 con 0x7f4a781039d0 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78113c90 con 0x7f4a78104390 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.254+0000 7f4a7d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78113f90 con 0x7f4a781099e0 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781039d0 0x7f4a7819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781039d0 0x7f4a7819a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51088/0 (socket says 172.21.15.94:51088) 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77400640 1 -- 172.21.15.94:0/1331682335 learned_addr learned my addr 172.21.15.94:0/1331682335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77e00640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 0x7f4a7819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a76a00640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a78104390 0x7f4a7819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77e00640 1 -- 172.21.15.94:0/1331682335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a78104390 msgr2=0x7f4a7819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77e00640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a78104390 0x7f4a7819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77e00640 1 -- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781039d0 msgr2=0x7f4a7819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77e00640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781039d0 0x7f4a7819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.255+0000 7f4a77e00640 1 -- 172.21.15.94:0/1331682335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7819f300 con 0x7f4a781099e0 2024-06-03T01:11:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.256+0000 7f4a77e00640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 0x7f4a7819ebd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000ec50 tx=0x7f4a7000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.256+0000 7f4a6fe00640 1 -- 172.21.15.94:0/1331682335 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70003b70 con 0x7f4a781099e0 2024-06-03T01:11:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.256+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7819f5f0 con 0x7f4a781099e0 2024-06-03T01:11:43.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.256+0000 7f4a6fe00640 1 -- 172.21.15.94:0/1331682335 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a70003d10 con 0x7f4a781099e0 2024-06-03T01:11:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.256+0000 7f4a6fe00640 1 -- 172.21.15.94:0/1331682335 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70010890 con 0x7f4a781099e0 2024-06-03T01:11:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.256+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a781a6e80 con 0x7f4a781099e0 2024-06-03T01:11:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.259+0000 7f4a6fe00640 1 -- 172.21.15.94:0/1331682335 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f4a70003fc0 con 0x7f4a781099e0 2024-06-03T01:11:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.260+0000 7f4a6fe00640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4a5c0a2e10 0x7f4a5c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.260+0000 7f4a813f4740 1 Processor -- start 2024-06-03T01:11:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a6fe00640 1 -- 172.21.15.94:0/1331682335 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f4a700cfff0 con 0x7f4a781099e0 2024-06-03T01:11:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a813f4740 1 -- start start 2024-06-03T01:11:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a77400640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4a5c0a2e10 0x7f4a5c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd63680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd78b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a813f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5607abcc7420 con 0x5607abd76790 2024-06-03T01:11:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a813f4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607abc6ec50 con 0x5607abd64770 2024-06-03T01:11:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.261+0000 7f4a813f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5607abcc33e0 con 0x5607abd63200 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd63680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd63680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:39026/0 (socket says 172.21.15.94:39026) 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 -- 172.21.15.94:0/2956048614 learned_addr learned my addr 172.21.15.94:0/2956048614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a77400640 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a77e00640 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 -- 172.21.15.94:0/2956048614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 msgr2=0x5607abd62cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd62cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 -- 172.21.15.94:0/2956048614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 msgr2=0x5607abd78b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a77400640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4a5c0a2e10 0x7f4a5c0a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4a680096c0 tx=0x7f4a68004040 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd78b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 -- 172.21.15.94:0/2956048614 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ab9db210 con 0x5607abd63200 2024-06-03T01:11:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a77e00640 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd78b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:11:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.262+0000 7f4a76a00640 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd63680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009cc0 tx=0x7f4a64011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6bc36ad02650d3a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.263+0000 7f4a6cc00640 1 -- 172.21.15.94:0/2956048614 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6401e070 con 0x5607abd63200 2024-06-03T01:11:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.263+0000 7f4a6cc00640 1 -- 172.21.15.94:0/2956048614 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a64019b10 con 0x5607abd63200 2024-06-03T01:11:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.263+0000 7f4a6cc00640 1 -- 172.21.15.94:0/2956048614 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400eaf0 con 0x5607abd63200 2024-06-03T01:11:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 -- 172.21.15.94:0/2956048614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 msgr2=0x5607abd63680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd63680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009cc0 tx=0x7f4a64011810 comp rx=0 tx=0).stop 2024-06-03T01:11:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 -- 172.21.15.94:0/2956048614 shutdown_connections 2024-06-03T01:11:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd78b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd63680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 --2- 172.21.15.94:0/2956048614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd62cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 -- 172.21.15.94:0/2956048614 >> 172.21.15.94:0/2956048614 conn(0x5607abd2bf80 msgr2=0x5607abd5f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 -- 172.21.15.94:0/2956048614 shutdown_connections 2024-06-03T01:11:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.264+0000 7f4a813f4740 1 -- 172.21.15.94:0/2956048614 wait complete. 2024-06-03T01:11:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.265+0000 7f4a813f4740 1 -- start start 2024-06-03T01:11:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.265+0000 7f4a813f4740 1 Processor -- start 2024-06-03T01:11:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd63200 0x5607abd737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd76790 0x5607abd74240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a813f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5607abd79560 con 0x5607abd63200 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a813f4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607abc820b0 con 0x5607abd64770 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a813f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5607abd796e0 con 0x5607abd76790 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a813f4740 10 client.0 _ll_register_callbacks cb 0x5607abc6df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a77e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd76790 0x5607abd74240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a77e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd76790 0x5607abd74240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:39034/0 (socket says 172.21.15.94:39034) 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a77e00640 1 -- 172.21.15.94:0/297533996 learned_addr learned my addr 172.21.15.94:0/297533996 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a77400640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd63200 0x5607abd737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.266+0000 7f4a76a00640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a76a00640 1 -- 172.21.15.94:0/297533996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd76790 msgr2=0x5607abd74240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a76a00640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd76790 0x5607abd74240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a76a00640 1 -- 172.21.15.94:0/297533996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd63200 msgr2=0x5607abd737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a76a00640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd63200 0x5607abd737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a76a00640 1 -- 172.21.15.94:0/297533996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607abaa7620 con 0x5607abd64770 2024-06-03T01:11:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a77e00640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd76790 0x5607abd74240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a76a00640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd73d00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400e6d0 tx=0x7f4a64009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b3d8d699f2b6f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.267+0000 7f4a61600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:11:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.268+0000 7f4a61600640 1 -- 172.21.15.94:0/297533996 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64027070 con 0x5607abd64770 2024-06-03T01:11:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.268+0000 7f4a61600640 1 -- 172.21.15.94:0/297533996 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f4a64019840 con 0x5607abd64770 2024-06-03T01:11:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.268+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a64019b10 con 0x5607abd64770 2024-06-03T01:11:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.268+0000 7f4a61600640 1 -- 172.21.15.94:0/297533996 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a6401e040 con 0x5607abd64770 2024-06-03T01:11:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.268+0000 7f4a813f4740 10 client.25403 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.268+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607abd85840 con 0x5607abd64770 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a61600640 1 -- 172.21.15.94:0/297533996 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a64019b10 con 0x5607abd64770 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a61600640 1 -- 172.21.15.94:0/297533996 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a6402a020 con 0x5607abd64770 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a813f4740 10 client.25403 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a813f4740 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a813f4740 20 client.25403 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a813f4740 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abd86c40 0x5607abd89030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a813f4740 4 client.25403 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607abcd4b50 con 0x5607abd86c40 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.269+0000 7f4a77400640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abd86c40 0x5607abd89030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.270+0000 7f4a77400640 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abd86c40 0x5607abd89030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=640f624b4bc0798f server_cookie=baa56610170cbeea in_seq=0 out_seq=0 2024-06-03T01:11:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.270+0000 7f4a61600640 10 client.25403 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a61600640 1 -- 172.21.15.94:0/297533996 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607abcd4b50 con 0x5607abd86c40 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a61600640 10 client.25403 handle_command_reply: tid=0 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 client.25403 shutdown 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 msgr2=0x5607abd73d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd73d00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400e6d0 tx=0x7f4a64009400 comp rx=0 tx=0).stop 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 shutdown_connections 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd76790 0x5607abd74240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abd86c40 0x5607abd89030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd73d00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 --2- 172.21.15.94:0/297533996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd63200 0x5607abd737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 >> 172.21.15.94:0/297533996 conn(0x5607abd2bf80 msgr2=0x5607abd65660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.271+0000 7f4a61600640 0 client.25403 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.272+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 shutdown_connections 2024-06-03T01:11:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.272+0000 7f4a813f4740 1 -- 172.21.15.94:0/297533996 wait complete. 2024-06-03T01:11:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.272+0000 7f4a813f4740 20 client.25403 trim_cache size 0 max 16384 2024-06-03T01:11:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.302+0000 7f4a813f4740 1 Processor -- start 2024-06-03T01:11:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.302+0000 7f4a813f4740 1 -- start start 2024-06-03T01:11:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd73e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd64770 0x5607abd5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd76790 0x5607abd5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a813f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781acab0 con 0x5607abd64770 2024-06-03T01:11:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a813f4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781ae5f0 con 0x5607abd76790 2024-06-03T01:11:43.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a813f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781ad180 con 0x5607abd63200 2024-06-03T01:11:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a76a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd64770 0x5607abd5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a76a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd64770 0x5607abd5f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51098/0 (socket says 172.21.15.94:51098) 2024-06-03T01:11:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a76a00640 1 -- 172.21.15.94:0/763435819 learned_addr learned my addr 172.21.15.94:0/763435819 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a77400640 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd73e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a77e00640 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd76790 0x5607abd5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a77400640 1 -- 172.21.15.94:0/763435819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd76790 msgr2=0x5607abd5f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a77400640 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd76790 0x5607abd5f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a77400640 1 -- 172.21.15.94:0/763435819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd64770 msgr2=0x5607abd5f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a77400640 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd64770 0x5607abd5f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.303+0000 7f4a77400640 1 -- 172.21.15.94:0/763435819 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607abd643b0 con 0x5607abd63200 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.304+0000 7f4a76a00640 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd64770 0x5607abd5f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.304+0000 7f4a77400640 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd73e90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a68031550 tx=0x7f4a6803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37484c55fa2eb64 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.304+0000 7f4a63e00640 1 -- 172.21.15.94:0/763435819 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68059070 con 0x5607abd63200 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.304+0000 7f4a63e00640 1 -- 172.21.15.94:0/763435819 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a680325c0 con 0x5607abd63200 2024-06-03T01:11:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.304+0000 7f4a63e00640 1 -- 172.21.15.94:0/763435819 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68035040 con 0x5607abd63200 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 -- 172.21.15.94:0/763435819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 msgr2=0x5607abd73e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd73e90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a68031550 tx=0x7f4a6803f000 comp rx=0 tx=0).stop 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 -- 172.21.15.94:0/763435819 shutdown_connections 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd76790 0x5607abd5f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd64770 0x5607abd5f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 --2- 172.21.15.94:0/763435819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd73e90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 -- 172.21.15.94:0/763435819 >> 172.21.15.94:0/763435819 conn(0x5607abd2bf80 msgr2=0x5607abd73780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 -- 172.21.15.94:0/763435819 shutdown_connections 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.305+0000 7f4a813f4740 1 -- 172.21.15.94:0/763435819 wait complete. 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 -- start start 2024-06-03T01:11:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 Processor -- start 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd79f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd7a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781adf20 con 0x5607abd76790 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781ad850 con 0x5607abd64770 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781aaa90 con 0x5607abd63200 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.306+0000 7f4a813f4740 10 client.0 _ll_register_callbacks cb 0x7f4a6401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd799d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:39050/0 (socket says 172.21.15.94:39050) 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 -- 172.21.15.94:0/3177230898 learned_addr learned my addr 172.21.15.94:0/3177230898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77e00640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd7a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a76a00640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd79f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 -- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 msgr2=0x5607abd79f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd79f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 -- 172.21.15.94:0/3177230898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 msgr2=0x5607abd7a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd7a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 -- 172.21.15.94:0/3177230898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607abdef9b0 con 0x5607abd63200 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a76a00640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd79f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a77400640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd799d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a6802ab80 tx=0x7f4a6802a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d4a8d3efe11273 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.307+0000 7f4a61600640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.308+0000 7f4a61600640 1 -- 172.21.15.94:0/3177230898 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68059040 con 0x5607abd63200 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.308+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a680325c0 con 0x5607abd63200 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.308+0000 7f4a61600640 1 -- 172.21.15.94:0/3177230898 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f4a68004a70 con 0x5607abd63200 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.308+0000 7f4a61600640 1 -- 172.21.15.94:0/3177230898 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a68004c70 con 0x5607abd63200 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.308+0000 7f4a813f4740 10 client.25423 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.308+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607abdf0cc0 con 0x5607abd63200 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a61600640 1 -- 172.21.15.94:0/3177230898 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a680325c0 con 0x5607abd63200 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a61600640 1 -- 172.21.15.94:0/3177230898 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4a6805d440 con 0x5607abd63200 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a813f4740 10 client.25423 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a813f4740 10 client.25423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a813f4740 20 client.25423 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a813f4740 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abdf1250 0x5607abdf3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a813f4740 4 client.25423 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607abcceea0 con 0x5607abdf1250 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.309+0000 7f4a76a00640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abdf1250 0x5607abdf3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.310+0000 7f4a76a00640 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abdf1250 0x5607abdf3670 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650208263cc37cd6 server_cookie=8c78ae6edc2794b8 in_seq=0 out_seq=0 2024-06-03T01:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.310+0000 7f4a61600640 10 client.25423 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.315+0000 7f4a61600640 1 -- 172.21.15.94:0/3177230898 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5607abcceea0 con 0x5607abdf1250 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.315+0000 7f4a61600640 10 client.25423 handle_command_reply: tid=0 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.315+0000 7f4a813f4740 1 client.25423 shutdown 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 msgr2=0x5607abd799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd799d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a6802ab80 tx=0x7f4a6802a440 comp rx=0 tx=0).stop 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 shutdown_connections 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607abd76790 0x5607abd7a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5607abdf1250 0x5607abdf3670 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607abd64770 0x5607abd79f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 --2- 172.21.15.94:0/3177230898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5607abd63200 0x5607abd799d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 >> 172.21.15.94:0/3177230898 conn(0x5607abd2bf80 msgr2=0x5607abd65660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a61600640 0 client.25423 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 shutdown_connections 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 1 -- 172.21.15.94:0/3177230898 wait complete. 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.316+0000 7f4a813f4740 20 client.25423 trim_cache size 0 max 16384 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:11:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:11:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4a5c0a2e10 msgr2=0x7f4a5c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4a5c0a2e10 0x7f4a5c0a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4a680096c0 tx=0x7f4a68004040 comp rx=0 tx=0).stop 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 msgr2=0x7f4a7819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 0x7f4a7819ebd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000ec50 tx=0x7f4a7000c6a0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 shutdown_connections 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4a5c0a2e10 0x7f4a5c0a52d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a781099e0 0x7f4a7819ebd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a78104390 0x7f4a7819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 --2- 172.21.15.94:0/1331682335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781039d0 0x7f4a7819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 >> 172.21.15.94:0/1331682335 conn(0x7f4a780ff480 msgr2=0x7f4a780ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 shutdown_connections 2024-06-03T01:11:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:43.317+0000 7f4a7d000640 1 -- 172.21.15.94:0/1331682335 wait complete. 2024-06-03T01:11:44.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:43 smithi094 ceph-mon[32119]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:44.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:43 smithi179 ceph-mon[37041]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:44.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:43 smithi107 ceph-mon[36622]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-03T01:11:46.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:45 smithi094 ceph-mon[32119]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:46.104 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:45 smithi179 ceph-mon[37041]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:46.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:45 smithi107 ceph-mon[36622]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:48.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:47 smithi094 ceph-mon[32119]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:47 smithi179 ceph-mon[37041]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:48.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:47 smithi107 ceph-mon[36622]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-03T01:11:50.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:49 smithi094 ceph-mon[32119]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-03T01:11:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:49 smithi179 ceph-mon[37041]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-03T01:11:50.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:49 smithi107 ceph-mon[36622]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-06-03T01:11:50.663 INFO:tasks.workunit.client.0.smithi094.stderr:Updating files: 63% (7971/12565) Updating files: 64% (8042/12565) Updating files: 65% (8168/12565) Updating files: 66% (8293/12565) Updating files: 67% (8419/12565) Updating files: 68% (8545/12565) Updating files: 69% (8670/12565) Updating files: 70% (8796/12565) Updating files: 71% (8922/12565) Updating files: 72% (9047/12565) Updating files: 73% (9173/12565) Updating files: 74% (9299/12565) Updating files: 75% (9424/12565) Updating files: 76% (9550/12565) Updating files: 77% (9676/12565) Updating files: 78% (9801/12565) Updating files: 79% (9927/12565) Updating files: 80% (10052/12565) Updating files: 81% (10178/12565) Updating files: 82% (10304/12565) Updating files: 83% (10429/12565) Updating files: 84% (10555/12565) Updating files: 85% (10681/12565) Updating files: 86% (10806/12565) Updating files: 87% (10932/12565) Updating files: 88% (11058/12565) Updating files: 89% (11183/12565) Updating files: 90% (11309/12565) Updating files: 91% (11435/12565) Updating files: 92% (11560/12565) Updating files: 93% (11686/12565) Updating files: 94% (11812/12565) Updating files: 95% (11937/12565) Updating files: 96% (12063/12565) Updating files: 97% (12189/12565) Updating files: 98% (12314/12565) Updating files: 99% (12440/12565) Updating files: 100% (12565/12565) Updating files: 100% (12565/12565), done. 2024-06-03T01:11:51.487 INFO:tasks.workunit.client.0.smithi094.stderr:Note: switching to 'dfcc4b3c2c55532c4c04fa47551c9df1dffdf746'. 2024-06-03T01:11:51.487 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-06-03T01:11:51.487 INFO:tasks.workunit.client.0.smithi094.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-03T01:11:51.487 INFO:tasks.workunit.client.0.smithi094.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-03T01:11:51.489 INFO:tasks.workunit.client.0.smithi094.stderr:state without impacting any branches by switching back to a branch. 2024-06-03T01:11:51.489 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-06-03T01:11:51.489 INFO:tasks.workunit.client.0.smithi094.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-03T01:11:51.489 INFO:tasks.workunit.client.0.smithi094.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-03T01:11:51.489 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-06-03T01:11:51.489 INFO:tasks.workunit.client.0.smithi094.stderr: git switch -c 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr:Or undo this operation with: 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr: git switch - 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr: 2024-06-03T01:11:51.490 INFO:tasks.workunit.client.0.smithi094.stderr:HEAD is now at dfcc4b3c2c5 Merge pull request #57814 from zdover23/wip-doc-2024-06-01-backport-57804-to-squid 2024-06-03T01:11:51.498 DEBUG:teuthology.orchestra.run.smithi094:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-03T01:11:51.600 INFO:tasks.workunit.client.0.smithi094.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-03T01:11:51.607 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-03T01:11:51.607 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-03T01:11:52.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:51 smithi094 ceph-mon[32119]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-06-03T01:11:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:51 smithi179 ceph-mon[37041]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-06-03T01:11:52.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:51 smithi107 ceph-mon[36622]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-06-03T01:11:52.521 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-03T01:11:52.633 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-03T01:11:52.722 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-03T01:11:52.729 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-03T01:11:52.729 INFO:tasks.workunit.client.0.smithi094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-03T01:11:52.806 INFO:tasks.workunit.client.0.smithi094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-03T01:11:52.815 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T01:11:52.815 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-03T01:11:52.865 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-06-03T01:11:52.866 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-06-03T01:11:52.867 DEBUG:teuthology.orchestra.run.smithi094:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-06-03T01:11:52.928 INFO:tasks.workunit.client.0.smithi094.stderr:+ dbench 1 2024-06-03T01:11:52.944 INFO:tasks.workunit.client.0.smithi094.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-03T01:11:52.944 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-06-03T01:11:52.944 INFO:tasks.workunit.client.0.smithi094.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-03T01:11:52.952 INFO:tasks.workunit.client.0.smithi094.stdout:0 of 1 processes prepared for launch 0 sec 2024-06-03T01:11:53.052 INFO:tasks.workunit.client.0.smithi094.stdout:1 of 1 processes prepared for launch 0 sec 2024-06-03T01:11:53.052 INFO:tasks.workunit.client.0.smithi094.stdout:releasing clients 2024-06-03T01:11:54.052 INFO:tasks.workunit.client.0.smithi094.stdout: 1 1915 26.45 MB/sec warmup 1 sec latency 12.753 ms 2024-06-03T01:11:54.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:53 smithi094 ceph-mon[32119]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-06-03T01:11:54.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:53 smithi094 ceph-mon[32119]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-06-03T01:11:54.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:53 smithi179 ceph-mon[37041]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-06-03T01:11:54.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:53 smithi179 ceph-mon[37041]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-06-03T01:11:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:53 smithi107 ceph-mon[36622]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-06-03T01:11:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:53 smithi107 ceph-mon[36622]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-06-03T01:11:55.052 INFO:tasks.workunit.client.0.smithi094.stdout: 1 4099 20.04 MB/sec warmup 2 sec latency 11.398 ms 2024-06-03T01:11:56.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 6434 17.10 MB/sec warmup 3 sec latency 11.104 ms 2024-06-03T01:11:56.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:55 smithi094 ceph-mon[32119]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-03T01:11:56.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:55 smithi179 ceph-mon[37041]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-03T01:11:56.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:55 smithi107 ceph-mon[36622]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 -- 172.21.15.94:0/802682195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 msgr2=0x7fe0e8102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 --2- 172.21.15.94:0/802682195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 0x7fe0e8102d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009900 tx=0x7fe0dc02a490 comp rx=0 tx=0).stop 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 -- 172.21.15.94:0/802682195 shutdown_connections 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 --2- 172.21.15.94:0/802682195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e810bd80 0x7fe0e810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 --2- 172.21.15.94:0/802682195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0e8103280 0x7fe0e810b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 --2- 172.21.15.94:0/802682195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 0x7fe0e8102d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 -- 172.21.15.94:0/802682195 >> 172.21.15.94:0/802682195 conn(0x7fe0e80fc1c0 msgr2=0x7fe0e80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 -- 172.21.15.94:0/802682195 shutdown_connections 2024-06-03T01:11:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.466+0000 7fe0ef000640 1 -- 172.21.15.94:0/802682195 wait complete. 2024-06-03T01:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.467+0000 7fe0ef000640 1 Processor -- start 2024-06-03T01:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.467+0000 7fe0ef000640 1 -- start start 2024-06-03T01:11:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ef000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 0x7fe0e81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ef000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e8103280 0x7fe0e8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ef000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0e810bd80 0x7fe0e819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ef000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8111bd0 con 0x7fe0e8103280 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ef000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8111a50 con 0x7fe0e810bd80 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ef000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8111d50 con 0x7fe0e8102960 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e8103280 0x7fe0e8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0e810bd80 0x7fe0e819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0edc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 0x7fe0e81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e8103280 0x7fe0e8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50998/0 (socket says 172.21.15.94:50998) 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.468+0000 7fe0ed200640 1 -- 172.21.15.94:0/1040566715 learned_addr learned my addr 172.21.15.94:0/1040566715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0edc00640 1 -- 172.21.15.94:0/1040566715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0e810bd80 msgr2=0x7fe0e819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0edc00640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0e810bd80 0x7fe0e819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0edc00640 1 -- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e8103280 msgr2=0x7fe0e8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0edc00640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e8103280 0x7fe0e8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0edc00640 1 -- 172.21.15.94:0/1040566715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e819d070 con 0x7fe0e8102960 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0ed200640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e8103280 0x7fe0e8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0edc00640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 0x7fe0e81980d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc02a9b0 tx=0x7fe0dc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0e6000640 1 -- 172.21.15.94:0/1040566715 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc03f070 con 0x7fe0e8102960 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0e6000640 1 -- 172.21.15.94:0/1040566715 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0dc031c90 con 0x7fe0e8102960 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e819d300 con 0x7fe0e8102960 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0e6000640 1 -- 172.21.15.94:0/1040566715 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc035030 con 0x7fe0e8102960 2024-06-03T01:11:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.469+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81a4bf0 con 0x7fe0e8102960 2024-06-03T01:11:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.473+0000 7fe0e6000640 1 -- 172.21.15.94:0/1040566715 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe0dc042070 con 0x7fe0e8102960 2024-06-03T01:11:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.473+0000 7fe0f35af740 1 Processor -- start 2024-06-03T01:11:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.473+0000 7fe0f35af740 1 -- start start 2024-06-03T01:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e283770 0x55e44e281cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e282680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e28cf80 0x55e44e28f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0f35af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e44e1e6420 con 0x55e44e28cf80 2024-06-03T01:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0f35af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e44e18dc50 con 0x55e44e282200 2024-06-03T01:11:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0f35af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e44e1e23e0 con 0x55e44e283770 2024-06-03T01:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e283770 0x55e44e281cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0edc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e282680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e28cf80 0x55e44e28f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e283770 0x55e44e281cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:47326/0 (socket says 172.21.15.94:47326) 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0ee600640 1 -- 172.21.15.94:0/1215669340 learned_addr learned my addr 172.21.15.94:0/1215669340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.474+0000 7fe0ee600640 1 -- 172.21.15.94:0/1215669340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 msgr2=0x55e44e282680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0ee600640 1 --2- 172.21.15.94:0/1215669340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e282680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0ee600640 1 -- 172.21.15.94:0/1215669340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e28cf80 msgr2=0x55e44e28f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0ee600640 1 --2- 172.21.15.94:0/1215669340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e28cf80 0x55e44e28f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0ee600640 1 -- 172.21.15.94:0/1215669340 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44defa210 con 0x55e44e283770 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0e6000640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0d00a2cf0 0x7fe0d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0ee600640 1 --2- 172.21.15.94:0/1215669340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e283770 0x55e44e281cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009fa0 tx=0x7fe0e0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fbd0129843b46c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0edc00640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0d00a2cf0 0x7fe0d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0daa00640 1 -- 172.21.15.94:0/1215669340 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0016070 con 0x55e44e283770 2024-06-03T01:11:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0e6000640 1 -- 172.21.15.94:0/1040566715 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe0dc0efba0 con 0x7fe0e8102960 2024-06-03T01:11:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0daa00640 1 -- 172.21.15.94:0/1215669340 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e0004d20 con 0x55e44e283770 2024-06-03T01:11:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.475+0000 7fe0daa00640 1 -- 172.21.15.94:0/1215669340 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0012840 con 0x55e44e283770 2024-06-03T01:11:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 -- 172.21.15.94:0/1215669340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e283770 msgr2=0x55e44e281cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 --2- 172.21.15.94:0/1215669340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e283770 0x55e44e281cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009fa0 tx=0x7fe0e0010740 comp rx=0 tx=0).stop 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0edc00640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0d00a2cf0 0x7fe0d00a51b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55e44e284ed0 tx=0x7fe0dc0f3650 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 -- 172.21.15.94:0/1215669340 shutdown_connections 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 --2- 172.21.15.94:0/1215669340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e28cf80 0x55e44e28f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 --2- 172.21.15.94:0/1215669340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e282680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 --2- 172.21.15.94:0/1215669340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e283770 0x55e44e281cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 -- 172.21.15.94:0/1215669340 >> 172.21.15.94:0/1215669340 conn(0x55e44e24af80 msgr2=0x55e44e27e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 -- 172.21.15.94:0/1215669340 shutdown_connections 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.476+0000 7fe0f35af740 1 -- 172.21.15.94:0/1215669340 wait complete. 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 -- start start 2024-06-03T01:11:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 Processor -- start 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e28a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e28a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e28aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e44e28fd50 con 0x55e44e283770 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e44e1a10b0 con 0x55e44e282200 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e44e28fed0 con 0x55e44e28cf80 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0f35af740 10 client.0 _ll_register_callbacks cb 0x55e44e18cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.477+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e28a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e28a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e28a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:55874/0 (socket says 172.21.15.94:55874) 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ee600640 1 -- 172.21.15.94:0/2145859218 learned_addr learned my addr 172.21.15.94:0/2145859218 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ee600640 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e28aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ed200640 1 -- 172.21.15.94:0/2145859218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 msgr2=0x55e44e28aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ed200640 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e28aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ed200640 1 -- 172.21.15.94:0/2145859218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 msgr2=0x55e44e28a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ed200640 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e28a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ed200640 1 -- 172.21.15.94:0/2145859218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44dfc6620 con 0x55e44e283770 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0ed200640 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e28a540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55e44e282ca0 tx=0x7fe0d4011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733e5e7cba630597 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.478+0000 7fe0cf400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0cf400640 1 -- 172.21.15.94:0/2145859218 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4004260 con 0x55e44e283770 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0cf400640 1 -- 172.21.15.94:0/2145859218 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe0d4004400 con 0x55e44e283770 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e0004d20 con 0x55e44e283770 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0cf400640 1 -- 172.21.15.94:0/2145859218 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0d4010960 con 0x55e44e283770 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0f35af740 10 client.15297 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e44e29c180 con 0x55e44e283770 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0cf400640 1 -- 172.21.15.94:0/2145859218 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e0004d20 con 0x55e44e283770 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.479+0000 7fe0cf400640 1 -- 172.21.15.94:0/2145859218 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0d401b420 con 0x55e44e283770 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0f35af740 10 client.15297 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0f35af740 10 client.15297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0f35af740 20 client.15297 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e29d640 0x55e44e29fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0f35af740 4 client.15297 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e44e1f3b50 con 0x55e44e29d640 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0ee600640 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e29d640 0x55e44e29fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0ee600640 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e29d640 0x55e44e29fa30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4fe8f4c94ec5875 server_cookie=9f3df30bbb49d45b in_seq=0 out_seq=0 2024-06-03T01:11:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.480+0000 7fe0cf400640 10 client.15297 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0cf400640 1 -- 172.21.15.94:0/2145859218 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e44e1f3b50 con 0x55e44e29d640 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0cf400640 10 client.15297 handle_command_reply: tid=0 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 client.15297 shutdown 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 msgr2=0x55e44e28a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e28a540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55e44e282ca0 tx=0x7fe0d4011c70 comp rx=0 tx=0).stop 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 shutdown_connections 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e28aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e28a540 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e29d640 0x55e44e29fa30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2145859218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e28a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0cf400640 0 client.15297 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.483+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 >> 172.21.15.94:0/2145859218 conn(0x55e44e24af80 msgr2=0x55e44e284660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.484+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 shutdown_connections 2024-06-03T01:11:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.484+0000 7fe0f35af740 1 -- 172.21.15.94:0/2145859218 wait complete. 2024-06-03T01:11:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.484+0000 7fe0f35af740 20 client.15297 trim_cache size 0 max 16384 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.501+0000 7fe0f35af740 1 Processor -- start 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.501+0000 7fe0f35af740 1 -- start start 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.501+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e282200 0x55e44e28a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e283770 0x55e44e2a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e2a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0f35af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81aec40 con 0x55e44e282200 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0f35af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b0780 con 0x55e44e283770 2024-06-03T01:11:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0f35af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81af310 con 0x55e44e28cf80 2024-06-03T01:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e282200 0x55e44e28a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e282200 0x55e44e28a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51014/0 (socket says 172.21.15.94:51014) 2024-06-03T01:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 -- 172.21.15.94:0/2675805047 learned_addr learned my addr 172.21.15.94:0/2675805047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ed200640 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e283770 0x55e44e2a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e2a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 -- 172.21.15.94:0/2675805047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 msgr2=0x55e44e2a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e2a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 -- 172.21.15.94:0/2675805047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e283770 msgr2=0x55e44e2a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e283770 0x55e44e2a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.502+0000 7fe0ee600640 1 -- 172.21.15.94:0/2675805047 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44e282cd0 con 0x55e44e282200 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.503+0000 7fe0ee600640 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e282200 0x55e44e28a6d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55e44e27e230 tx=0x7fe0e0009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b114a2c2bf59a60 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.503+0000 7fe0da000640 1 -- 172.21.15.94:0/2675805047 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0016040 con 0x55e44e282200 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.503+0000 7fe0da000640 1 -- 172.21.15.94:0/2675805047 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e002cab0 con 0x55e44e282200 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.503+0000 7fe0da000640 1 -- 172.21.15.94:0/2675805047 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0012dd0 con 0x55e44e282200 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.503+0000 7fe0f35af740 1 -- 172.21.15.94:0/2675805047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e282200 msgr2=0x55e44e28a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.503+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e282200 0x55e44e28a6d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55e44e27e230 tx=0x7fe0e0009600 comp rx=0 tx=0).stop 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 -- 172.21.15.94:0/2675805047 shutdown_connections 2024-06-03T01:11:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e2a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e283770 0x55e44e2a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 --2- 172.21.15.94:0/2675805047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e282200 0x55e44e28a6d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 -- 172.21.15.94:0/2675805047 >> 172.21.15.94:0/2675805047 conn(0x55e44e24af80 msgr2=0x55e44e289fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 -- 172.21.15.94:0/2675805047 shutdown_connections 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 -- 172.21.15.94:0/2675805047 wait complete. 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 -- start start 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.504+0000 7fe0f35af740 1 Processor -- start 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e290ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e291530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0f35af740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e291a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0f35af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81b00b0 con 0x55e44e283770 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0f35af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81af9e0 con 0x55e44e282200 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0f35af740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e81acc20 con 0x55e44e28cf80 2024-06-03T01:11:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0f35af740 10 client.0 _ll_register_callbacks cb 0x7fe0d400bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e290ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e290ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:55896/0 (socket says 172.21.15.94:55896) 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 -- 172.21.15.94:0/339990447 learned_addr learned my addr 172.21.15.94:0/339990447 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e291a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ed200640 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e291530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 -- 172.21.15.94:0/339990447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 msgr2=0x55e44e291a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e291a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 -- 172.21.15.94:0/339990447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 msgr2=0x55e44e291530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:11:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e291530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.505+0000 7fe0ee600640 1 -- 172.21.15.94:0/339990447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44e30b720 con 0x55e44e282200 2024-06-03T01:11:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0ee600640 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e290ff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0012450 tx=0x7fe0e00129b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92515ef9ef5448b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:11:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0cf400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0cf400640 1 -- 172.21.15.94:0/339990447 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0016070 con 0x55e44e282200 2024-06-03T01:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e002cab0 con 0x55e44e282200 2024-06-03T01:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0cf400640 1 -- 172.21.15.94:0/339990447 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe0e0012570 con 0x55e44e282200 2024-06-03T01:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0cf400640 1 -- 172.21.15.94:0/339990447 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0e0012b50 con 0x55e44e282200 2024-06-03T01:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0f35af740 10 client.25427 fetch_fsmap learned FSMap version 28 2024-06-03T01:11:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e44e30ca10 con 0x55e44e282200 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.506+0000 7fe0cf400640 1 -- 172.21.15.94:0/339990447 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e002cab0 con 0x55e44e282200 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0cf400640 1 -- 172.21.15.94:0/339990447 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0e0028040 con 0x55e44e282200 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0f35af740 10 client.25427 fetch_fsmap finished waiting for FSMap version 28 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0f35af740 10 client.25427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0f35af740 20 client.25427 populate_metadata read hostname 'smithi094' 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0f35af740 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e30ce90 0x55e44e30f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0f35af740 4 client.25427 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e44e1edea0 con 0x55e44e30ce90 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0ed200640 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e30ce90 0x55e44e30f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0cf400640 10 client.25427 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.507+0000 7fe0ed200640 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e30ce90 0x55e44e30f2b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7cb07f6e5c66bb4 server_cookie=e338ea03ce372ab8 in_seq=0 out_seq=0 2024-06-03T01:11:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0cf400640 1 -- 172.21.15.94:0/339990447 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e44e1edea0 con 0x55e44e30ce90 2024-06-03T01:11:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0cf400640 10 client.25427 handle_command_reply: tid=0 2024-06-03T01:11:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 client.25427 shutdown 2024-06-03T01:11:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 msgr2=0x55e44e290ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e290ff0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0012450 tx=0x7fe0e00129b0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 shutdown_connections 2024-06-03T01:11:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e44e28cf80 0x55e44e291a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e44e283770 0x55e44e291530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e44e30ce90 0x55e44e30f2b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 --2- 172.21.15.94:0/339990447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e44e282200 0x55e44e290ff0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0cf400640 0 client.25427 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:11:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 >> 172.21.15.94:0/339990447 conn(0x55e44e24af80 msgr2=0x55e44e28a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 shutdown_connections 2024-06-03T01:11:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 1 -- 172.21.15.94:0/339990447 wait complete. 2024-06-03T01:11:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.646+0000 7fe0f35af740 20 client.25427 trim_cache size 0 max 16384 2024-06-03T01:11:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:11:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:11:56.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:11:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0d00a2cf0 msgr2=0x7fe0d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0d00a2cf0 0x7fe0d00a51b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55e44e284ed0 tx=0x7fe0dc0f3650 comp rx=0 tx=0).stop 2024-06-03T01:11:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 msgr2=0x7fe0e81980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:11:56.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 0x7fe0e81980d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc02a9b0 tx=0x7fe0dc003b90 comp rx=0 tx=0).stop 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 shutdown_connections 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0d00a2cf0 0x7fe0d00a51b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0e810bd80 0x7fe0e819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0e8103280 0x7fe0e8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 --2- 172.21.15.94:0/1040566715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0e8102960 0x7fe0e81980d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 >> 172.21.15.94:0/1040566715 conn(0x7fe0e80fc1c0 msgr2=0x7fe0e80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 shutdown_connections 2024-06-03T01:11:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:11:56.647+0000 7fe0ef000640 1 -- 172.21.15.94:0/1040566715 wait complete. 2024-06-03T01:11:57.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 8028 15.00 MB/sec warmup 4 sec latency 48.570 ms 2024-06-03T01:11:58.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 10719 15.37 MB/sec warmup 5 sec latency 16.192 ms 2024-06-03T01:11:58.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:57 smithi094 ceph-mon[32119]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-03T01:11:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:57 smithi179 ceph-mon[37041]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-03T01:11:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:57 smithi107 ceph-mon[36622]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-06-03T01:11:59.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 12208 13.91 MB/sec warmup 6 sec latency 11.848 ms 2024-06-03T01:12:00.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 14590 13.92 MB/sec warmup 7 sec latency 8.769 ms 2024-06-03T01:12:00.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:11:59 smithi094 ceph-mon[32119]: pgmap v52: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 390 KiB/s wr, 136 op/s 2024-06-03T01:12:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:11:59 smithi179 ceph-mon[37041]: pgmap v52: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 390 KiB/s wr, 136 op/s 2024-06-03T01:12:00.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:11:59 smithi107 ceph-mon[36622]: pgmap v52: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 390 KiB/s wr, 136 op/s 2024-06-03T01:12:01.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 16459 12.91 MB/sec warmup 8 sec latency 51.371 ms 2024-06-03T01:12:02.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 16693 11.69 MB/sec warmup 9 sec latency 76.545 ms 2024-06-03T01:12:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:01 smithi179 ceph-mon[37041]: pgmap v53: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 389 KiB/s wr, 131 op/s 2024-06-03T01:12:02.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:01 smithi107 ceph-mon[36622]: pgmap v53: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 389 KiB/s wr, 131 op/s 2024-06-03T01:12:02.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:01 smithi094 ceph-mon[32119]: pgmap v53: 129 pgs: 129 active+clean; 5.2 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 389 KiB/s wr, 131 op/s 2024-06-03T01:12:03.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 18485 11.69 MB/sec warmup 10 sec latency 21.562 ms 2024-06-03T01:12:04.053 INFO:tasks.workunit.client.0.smithi094.stdout: 1 21116 11.91 MB/sec warmup 11 sec latency 16.089 ms 2024-06-03T01:12:04.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:03 smithi107 ceph-mon[36622]: pgmap v54: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 921 KiB/s wr, 238 op/s 2024-06-03T01:12:04.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:03 smithi094 ceph-mon[32119]: pgmap v54: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 921 KiB/s wr, 238 op/s 2024-06-03T01:12:04.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:03 smithi179 ceph-mon[37041]: pgmap v54: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 921 KiB/s wr, 238 op/s 2024-06-03T01:12:05.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 22450 11.48 MB/sec warmup 12 sec latency 13.072 ms 2024-06-03T01:12:06.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 25107 11.89 MB/sec warmup 13 sec latency 9.894 ms 2024-06-03T01:12:06.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:05 smithi107 ceph-mon[36622]: pgmap v55: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 920 KiB/s wr, 233 op/s 2024-06-03T01:12:06.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:05 smithi094 ceph-mon[32119]: pgmap v55: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 920 KiB/s wr, 233 op/s 2024-06-03T01:12:06.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:05 smithi179 ceph-mon[37041]: pgmap v55: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 920 KiB/s wr, 233 op/s 2024-06-03T01:12:07.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 26534 11.44 MB/sec warmup 14 sec latency 17.236 ms 2024-06-03T01:12:08.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 28803 11.60 MB/sec warmup 15 sec latency 16.182 ms 2024-06-03T01:12:08.152 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:07 smithi094 ceph-mon[32119]: pgmap v56: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 920 KiB/s wr, 233 op/s 2024-06-03T01:12:08.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:07 smithi107 ceph-mon[36622]: pgmap v56: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 920 KiB/s wr, 233 op/s 2024-06-03T01:12:08.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:07 smithi179 ceph-mon[37041]: pgmap v56: 129 pgs: 129 active+clean; 10 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 920 KiB/s wr, 233 op/s 2024-06-03T01:12:09.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 30847 11.32 MB/sec warmup 16 sec latency 12.895 ms 2024-06-03T01:12:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 -- 172.21.15.94:0/2047816138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd528108bb0 msgr2=0x7fd52810cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 --2- 172.21.15.94:0/2047816138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd528108bb0 0x7fd52810cbf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd514009900 tx=0x7fd51402a410 comp rx=0 tx=0).stop 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 -- 172.21.15.94:0/2047816138 shutdown_connections 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 --2- 172.21.15.94:0/2047816138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52810d2d0 0x7fd528114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 --2- 172.21.15.94:0/2047816138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd528108bb0 0x7fd52810cbf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 --2- 172.21.15.94:0/2047816138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 0x7fd5281085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 -- 172.21.15.94:0/2047816138 >> 172.21.15.94:0/2047816138 conn(0x7fd528075f30 msgr2=0x7fd528078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 -- 172.21.15.94:0/2047816138 shutdown_connections 2024-06-03T01:12:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.790+0000 7fd530600640 1 -- 172.21.15.94:0/2047816138 wait complete. 2024-06-03T01:12:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.791+0000 7fd530600640 1 Processor -- start 2024-06-03T01:12:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.791+0000 7fd530600640 1 -- start start 2024-06-03T01:12:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd530600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 0x7fd5281a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd530600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52810d2d0 0x7fd5281a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd530600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd52819e600 0x7fd52819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd530600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd528118360 con 0x7fd52819e600 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd530600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281181e0 con 0x7fd528108200 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd530600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281184e0 con 0x7fd52810d2d0 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 0x7fd5281a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 0x7fd5281a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56470/0 (socket says 172.21.15.94:56470) 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52dc00640 1 -- 172.21.15.94:0/2950028496 learned_addr learned my addr 172.21.15.94:0/2950028496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52e600640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd52819e600 0x7fd52819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52d200640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52810d2d0 0x7fd5281a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52dc00640 1 -- 172.21.15.94:0/2950028496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52810d2d0 msgr2=0x7fd5281a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52dc00640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52810d2d0 0x7fd5281a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52dc00640 1 -- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd52819e600 msgr2=0x7fd52819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.792+0000 7fd52dc00640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd52819e600 0x7fd52819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.793+0000 7fd52dc00640 1 -- 172.21.15.94:0/2950028496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52819f120 con 0x7fd528108200 2024-06-03T01:12:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.793+0000 7fd52dc00640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 0x7fd5281a4400 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e9f0 tx=0x7fd51c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.793+0000 7fd526000640 1 -- 172.21.15.94:0/2950028496 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00cd80 con 0x7fd528108200 2024-06-03T01:12:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.793+0000 7fd526000640 1 -- 172.21.15.94:0/2950028496 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51c010910 con 0x7fd528108200 2024-06-03T01:12:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.793+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52819f410 con 0x7fd528108200 2024-06-03T01:12:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.793+0000 7fd526000640 1 -- 172.21.15.94:0/2950028496 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c018990 con 0x7fd528108200 2024-06-03T01:12:09.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.793+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5281ab610 con 0x7fd528108200 2024-06-03T01:12:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.796+0000 7fd526000640 1 -- 172.21.15.94:0/2950028496 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd51c010ac0 con 0x7fd528108200 2024-06-03T01:12:09.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 Processor -- start 2024-06-03T01:12:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 -- start start 2024-06-03T01:12:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bb3672a420 con 0x55bb367c75b0 2024-06-03T01:12:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bb366d1c50 con 0x55bb367c6040 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd53343c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bb367263e0 con 0x55bb367d0dc0 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd52dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd52e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd52dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56482/0 (socket says 172.21.15.94:56482) 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.797+0000 7fd52dc00640 1 -- 172.21.15.94:0/2300453342 learned_addr learned my addr 172.21.15.94:0/2300453342 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52d200640 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52e600640 1 -- 172.21.15.94:0/2300453342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 msgr2=0x55bb367d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52e600640 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52e600640 1 -- 172.21.15.94:0/2300453342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 msgr2=0x55bb367c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52e600640 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52e600640 1 -- 172.21.15.94:0/2300453342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3643e210 con 0x55bb367c75b0 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52dc00640 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd52e600640 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367c5b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd520009f00 tx=0x7fd520011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2c8404206dee69 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.798+0000 7fd526000640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd5100a2da0 0x7fd5100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd51aa00640 1 -- 172.21.15.94:0/2300453342 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520004d20 con 0x55bb367c75b0 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd51aa00640 1 -- 172.21.15.94:0/2300453342 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd520004050 con 0x55bb367c75b0 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd51aa00640 1 -- 172.21.15.94:0/2300453342 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520005260 con 0x55bb367c75b0 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd52dc00640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd5100a2da0 0x7fd5100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd526000640 1 -- 172.21.15.94:0/2950028496 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fd51c0cfbd0 con 0x7fd528108200 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd53343c740 1 -- 172.21.15.94:0/2300453342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 msgr2=0x55bb367c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd53343c740 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367c5b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd520009f00 tx=0x7fd520011810 comp rx=0 tx=0).stop 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.799+0000 7fd52dc00640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd5100a2da0 0x7fd5100a5260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e9c0 tx=0x7fd51c0d3680 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 -- 172.21.15.94:0/2300453342 shutdown_connections 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 --2- 172.21.15.94:0/2300453342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367c5b00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 -- 172.21.15.94:0/2300453342 >> 172.21.15.94:0/2300453342 conn(0x55bb3678ef80 msgr2=0x55bb367c2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 -- 172.21.15.94:0/2300453342 shutdown_connections 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 -- 172.21.15.94:0/2300453342 wait complete. 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 -- start start 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.800+0000 7fd53343c740 1 Processor -- start 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd53343c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd53343c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd53343c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd53343c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bb367d3b90 con 0x55bb367c75b0 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd53343c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bb366e50b0 con 0x55bb367c6040 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd53343c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bb367d3d10 con 0x55bb367d0dc0 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd53343c740 10 client.0 _ll_register_callbacks cb 0x55bb366d0f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd52e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd52e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56498/0 (socket says 172.21.15.94:56498) 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd52e600640 1 -- 172.21.15.94:0/2454399930 learned_addr learned my addr 172.21.15.94:0/2454399930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:09.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd52e600640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.801+0000 7fd52d200640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd52e600640 1 -- 172.21.15.94:0/2454399930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 msgr2=0x55bb367ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd52e600640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd52e600640 1 -- 172.21.15.94:0/2454399930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 msgr2=0x55bb367ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd52e600640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd52e600640 1 -- 172.21.15.94:0/2454399930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3650a620 con 0x55bb367c6040 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd52d200640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd52e600640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367cdde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd520009d40 tx=0x7fd5200048c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a21bd6251ab8906 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.802+0000 7fd50f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd50f400640 1 -- 172.21.15.94:0/2454399930 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520028480 con 0x55bb367c6040 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd50f400640 1 -- 172.21.15.94:0/2454399930 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fd520028620 con 0x55bb367c6040 2024-06-03T01:12:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd520004050 con 0x55bb367c6040 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd50f400640 1 -- 172.21.15.94:0/2454399930 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd520036050 con 0x55bb367c6040 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd53343c740 10 client.25439 fetch_fsmap learned FSMap version 29 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb367dff00 con 0x55bb367c6040 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd50f400640 1 -- 172.21.15.94:0/2454399930 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd520004050 con 0x55bb367c6040 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.803+0000 7fd50f400640 1 -- 172.21.15.94:0/2454399930 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd520025040 con 0x55bb367c6040 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd53343c740 10 client.25439 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd53343c740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd53343c740 20 client.25439 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd53343c740 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb367e1300 0x55bb367e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd53343c740 4 client.25439 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb36737b50 con 0x55bb367e1300 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd52d200640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb367e1300 0x55bb367e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd52d200640 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb367e1300 0x55bb367e36f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e19b5b85ba894c server_cookie=b09c2b57f75551c9 in_seq=0 out_seq=0 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.804+0000 7fd50f400640 10 client.25439 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.805+0000 7fd50f400640 1 -- 172.21.15.94:0/2454399930 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb36737b50 con 0x55bb367e1300 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.805+0000 7fd50f400640 10 client.25439 handle_command_reply: tid=0 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.805+0000 7fd53343c740 1 client.25439 shutdown 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.805+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 msgr2=0x55bb367cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.805+0000 7fd53343c740 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367cdde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd520009d40 tx=0x7fd5200048c0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.805+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 shutdown_connections 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.805+0000 7fd53343c740 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd53343c740 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c75b0 0x55bb367ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd53343c740 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb367e1300 0x55bb367e36f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd53343c740 1 --2- 172.21.15.94:0/2454399930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367cdde0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 >> 172.21.15.94:0/2454399930 conn(0x55bb3678ef80 msgr2=0x55bb367c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd50f400640 0 client.25439 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 shutdown_connections 2024-06-03T01:12:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd53343c740 1 -- 172.21.15.94:0/2454399930 wait complete. 2024-06-03T01:12:09.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.806+0000 7fd53343c740 20 client.25439 trim_cache size 0 max 16384 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.822+0000 7fd53343c740 1 Processor -- start 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd53343c740 1 -- start start 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd53343c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd53343c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367c75b0 0x55bb367e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd53343c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367d0dc0 0x55bb367e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd53343c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281b52e0 con 0x55bb367d0dc0 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd53343c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281b6e50 con 0x55bb367c6040 2024-06-03T01:12:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd53343c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281b59b0 con 0x55bb367c75b0 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd52e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd52d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367c75b0 0x55bb367e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd52d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367c75b0 0x55bb367e8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55844/0 (socket says 172.21.15.94:55844) 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd52d200640 1 -- 172.21.15.94:0/1366446067 learned_addr learned my addr 172.21.15.94:0/1366446067 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.823+0000 7fd52e600640 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367d0dc0 0x55bb367e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd52d200640 1 -- 172.21.15.94:0/1366446067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 msgr2=0x55bb367c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd52d200640 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd52d200640 1 -- 172.21.15.94:0/1366446067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367d0dc0 msgr2=0x55bb367e93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd52d200640 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367d0dc0 0x55bb367e93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd52d200640 1 -- 172.21.15.94:0/1366446067 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb367d3610 con 0x55bb367c75b0 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd52e600640 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367d0dc0 0x55bb367e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd52d200640 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367c75b0 0x55bb367e8e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd514005940 tx=0x7fd5140058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecae829637484645 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd51a000640 1 -- 172.21.15.94:0/1366446067 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514035040 con 0x55bb367c75b0 2024-06-03T01:12:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd51a000640 1 -- 172.21.15.94:0/1366446067 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51404a070 con 0x55bb367c75b0 2024-06-03T01:12:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.824+0000 7fd51a000640 1 -- 172.21.15.94:0/1366446067 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140046c0 con 0x55bb367c75b0 2024-06-03T01:12:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 -- 172.21.15.94:0/1366446067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367c75b0 msgr2=0x55bb367e8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367c75b0 0x55bb367e8e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd514005940 tx=0x7fd5140058b0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 -- 172.21.15.94:0/1366446067 shutdown_connections 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367d0dc0 0x55bb367e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367c75b0 0x55bb367e8e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 --2- 172.21.15.94:0/1366446067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c6040 0x55bb367c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 -- 172.21.15.94:0/1366446067 >> 172.21.15.94:0/1366446067 conn(0x55bb3678ef80 msgr2=0x55bb367c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 -- 172.21.15.94:0/1366446067 shutdown_connections 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 -- 172.21.15.94:0/1366446067 wait complete. 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.825+0000 7fd53343c740 1 -- start start 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 1 Processor -- start 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c6040 0x55bb367d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c75b0 0x55bb367d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281b6750 con 0x55bb367c6040 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281b6080 con 0x55bb367c75b0 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd5281b32c0 con 0x55bb367d0dc0 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd53343c740 10 client.0 _ll_register_callbacks cb 0x7fd51c0d3550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd52e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c6040 0x55bb367d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd52e600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c6040 0x55bb367d96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51366/0 (socket says 172.21.15.94:51366) 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd52e600640 1 -- 172.21.15.94:0/1126307291 learned_addr learned my addr 172.21.15.94:0/1126307291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.826+0000 7fd52d200640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c75b0 0x55bb367d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52e600640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52d200640 1 -- 172.21.15.94:0/1126307291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 msgr2=0x55bb367d5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52d200640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52d200640 1 -- 172.21.15.94:0/1126307291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c6040 msgr2=0x55bb367d96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52d200640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c6040 0x55bb367d96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52d200640 1 -- 172.21.15.94:0/1126307291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb367d6730 con 0x55bb367c75b0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52e600640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c6040 0x55bb367d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd52d200640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c75b0 0x55bb367d5a20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55bb367bdf70 tx=0x7fd514032b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bdf3bbb8a19fe01 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd50f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd50f400640 1 -- 172.21.15.94:0/1126307291 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514052020 con 0x55bb367c75b0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd50f400640 1 -- 172.21.15.94:0/1126307291 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fd514005250 con 0x55bb367c75b0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.827+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd51404a070 con 0x55bb367c75b0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd50f400640 1 -- 172.21.15.94:0/1126307291 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd51405d050 con 0x55bb367c75b0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 10 client.25445 fetch_fsmap learned FSMap version 29 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb3684ade0 con 0x55bb367c75b0 2024-06-03T01:12:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd50f400640 1 -- 172.21.15.94:0/1126307291 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51404a070 con 0x55bb367c75b0 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd50f400640 1 -- 172.21.15.94:0/1126307291 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd5140035f0 con 0x55bb367c75b0 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 10 client.25445 fetch_fsmap finished waiting for FSMap version 29 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 10 client.25445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 20 client.25445 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb3684b550 0x55bb3684d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 4 client.25445 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb36731ea0 con 0x55bb3684b550 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.828+0000 7fd52e600640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb3684b550 0x55bb3684d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.829+0000 7fd52e600640 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb3684b550 0x55bb3684d970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f396d61c9931599 server_cookie=ad0058b7af824a30 in_seq=0 out_seq=0 2024-06-03T01:12:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:09.829+0000 7fd50f400640 10 client.25445 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd50f400640 1 -- 172.21.15.94:0/1126307291 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb36731ea0 con 0x55bb3684b550 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd50f400640 10 client.25445 handle_command_reply: tid=0 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd53343c740 1 client.25445 shutdown 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c75b0 msgr2=0x55bb367d5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd53343c740 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c75b0 0x55bb367d5a20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55bb367bdf70 tx=0x7fd514032b10 comp rx=0 tx=0).stop 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 shutdown_connections 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd53343c740 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb367d0dc0 0x55bb367d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.003+0000 7fd53343c740 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb3684b550 0x55bb3684d970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.004+0000 7fd53343c740 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb367c75b0 0x55bb367d5a20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.004+0000 7fd50f400640 0 client.25445 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.004+0000 7fd53343c740 1 --2- 172.21.15.94:0/1126307291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb367c6040 0x55bb367d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.004+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 >> 172.21.15.94:0/1126307291 conn(0x55bb3678ef80 msgr2=0x55bb367c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.004+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 shutdown_connections 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.004+0000 7fd53343c740 1 -- 172.21.15.94:0/1126307291 wait complete. 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.004+0000 7fd53343c740 20 client.25445 trim_cache size 0 max 16384 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:12:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:12:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd5100a2da0 msgr2=0x7fd5100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd5100a2da0 0x7fd5100a5260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e9c0 tx=0x7fd51c0d3680 comp rx=0 tx=0).stop 2024-06-03T01:12:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 msgr2=0x7fd5281a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 0x7fd5281a4400 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e9f0 tx=0x7fd51c00eeb0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 shutdown_connections 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd5100a2da0 0x7fd5100a5260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd52819e600 0x7fd52819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd52810d2d0 0x7fd5281a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 --2- 172.21.15.94:0/2950028496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528108200 0x7fd5281a4400 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 >> 172.21.15.94:0/2950028496 conn(0x7fd528075f30 msgr2=0x7fd528076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 shutdown_connections 2024-06-03T01:12:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:10.005+0000 7fd530600640 1 -- 172.21.15.94:0/2950028496 wait complete. 2024-06-03T01:12:10.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32203 11.34 MB/sec warmup 17 sec latency 379.066 ms 2024-06-03T01:12:10.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:09 smithi107 ceph-mon[36622]: pgmap v57: 129 pgs: 129 active+clean; 16 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.5 MiB/s wr, 298 op/s 2024-06-03T01:12:10.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:09 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:10.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:09 smithi094 ceph-mon[32119]: pgmap v57: 129 pgs: 129 active+clean; 16 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.5 MiB/s wr, 298 op/s 2024-06-03T01:12:10.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:09 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:10.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:09 smithi179 ceph-mon[37041]: pgmap v57: 129 pgs: 129 active+clean; 16 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.5 MiB/s wr, 298 op/s 2024-06-03T01:12:10.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:09 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:11.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32207 10.71 MB/sec warmup 18 sec latency 1370.499 ms 2024-06-03T01:12:12.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32686 10.17 MB/sec warmup 19 sec latency 412.549 ms 2024-06-03T01:12:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:11 smithi107 ceph-mon[36622]: pgmap v58: 129 pgs: 129 active+clean; 16 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-03T01:12:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:11 smithi094 ceph-mon[32119]: pgmap v58: 129 pgs: 129 active+clean; 16 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-03T01:12:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:11 smithi179 ceph-mon[37041]: pgmap v58: 129 pgs: 129 active+clean; 16 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-06-03T01:12:13.054 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32686 9.67 MB/sec warmup 20 sec latency 1412.643 ms 2024-06-03T01:12:14.055 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32686 9.21 MB/sec warmup 21 sec latency 2412.734 ms 2024-06-03T01:12:14.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:13 smithi107 ceph-mon[36622]: pgmap v59: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.5 MiB/s wr, 327 op/s 2024-06-03T01:12:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:13 smithi094 ceph-mon[32119]: pgmap v59: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.5 MiB/s wr, 327 op/s 2024-06-03T01:12:14.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:13 smithi179 ceph-mon[37041]: pgmap v59: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.5 MiB/s wr, 327 op/s 2024-06-03T01:12:15.055 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32686 8.79 MB/sec warmup 22 sec latency 3412.887 ms 2024-06-03T01:12:15.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:14 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:15.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:14 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:14 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:16.055 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32687 8.41 MB/sec warmup 23 sec latency 4406.027 ms 2024-06-03T01:12:16.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:15 smithi107 ceph-mon[36622]: pgmap v60: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 959 KiB/s wr, 220 op/s 2024-06-03T01:12:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:15 smithi094 ceph-mon[32119]: pgmap v60: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 959 KiB/s wr, 220 op/s 2024-06-03T01:12:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:15 smithi179 ceph-mon[37041]: pgmap v60: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 959 KiB/s wr, 220 op/s 2024-06-03T01:12:17.055 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32687 8.06 MB/sec warmup 24 sec latency 1007.125 ms 2024-06-03T01:12:18.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32687 7.73 MB/sec warmup 25 sec latency 2007.225 ms 2024-06-03T01:12:18.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:17 smithi107 ceph-mon[36622]: pgmap v61: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 959 KiB/s wr, 220 op/s 2024-06-03T01:12:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:17 smithi094 ceph-mon[32119]: pgmap v61: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 959 KiB/s wr, 220 op/s 2024-06-03T01:12:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:17 smithi179 ceph-mon[37041]: pgmap v61: 129 pgs: 129 active+clean; 11 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 959 KiB/s wr, 220 op/s 2024-06-03T01:12:19.055 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32687 7.44 MB/sec warmup 26 sec latency 3007.327 ms 2024-06-03T01:12:20.055 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32687 7.16 MB/sec warmup 27 sec latency 4007.456 ms 2024-06-03T01:12:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:19 smithi107 ceph-mon[36622]: pgmap v62: 129 pgs: 129 active+clean; 11 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 981 KiB/s wr, 234 op/s 2024-06-03T01:12:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:19 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:19 smithi094 ceph-mon[32119]: pgmap v62: 129 pgs: 129 active+clean; 11 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 981 KiB/s wr, 234 op/s 2024-06-03T01:12:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:19 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:19 smithi179 ceph-mon[37041]: pgmap v62: 129 pgs: 129 active+clean; 11 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 981 KiB/s wr, 234 op/s 2024-06-03T01:12:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:19 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:21.055 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32688 6.90 MB/sec warmup 28 sec latency 5004.790 ms 2024-06-03T01:12:22.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32754 6.67 MB/sec warmup 29 sec latency 952.029 ms 2024-06-03T01:12:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:21 smithi107 ceph-mon[36622]: pgmap v63: 129 pgs: 129 active+clean; 11 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 412 KiB/s wr, 169 op/s 2024-06-03T01:12:22.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:21 smithi094 ceph-mon[32119]: pgmap v63: 129 pgs: 129 active+clean; 11 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 412 KiB/s wr, 169 op/s 2024-06-03T01:12:22.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:21 smithi179 ceph-mon[37041]: pgmap v63: 129 pgs: 129 active+clean; 11 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 412 KiB/s wr, 169 op/s 2024-06-03T01:12:23.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32754 6.45 MB/sec warmup 30 sec latency 1952.128 ms 2024-06-03T01:12:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 -- 172.21.15.94:0/2537885509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e81047c0 msgr2=0x7f23e8108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 --2- 172.21.15.94:0/2537885509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e81047c0 0x7f23e8108800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009900 tx=0x7f23d002a3f0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 -- 172.21.15.94:0/2537885509 shutdown_connections 2024-06-03T01:12:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 --2- 172.21.15.94:0/2537885509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23e8108ee0 0x7f23e8110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 --2- 172.21.15.94:0/2537885509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e81047c0 0x7f23e8108800 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 --2- 172.21.15.94:0/2537885509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e8103e10 0x7f23e81041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 -- 172.21.15.94:0/2537885509 >> 172.21.15.94:0/2537885509 conn(0x7f23e80ff6d0 msgr2=0x7f23e8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 -- 172.21.15.94:0/2537885509 shutdown_connections 2024-06-03T01:12:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.146+0000 7f23ed200640 1 -- 172.21.15.94:0/2537885509 wait complete. 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 Processor -- start 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 -- start start 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23e8103e10 0x7f23e819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e8108ee0 0x7f23e819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e819ed00 0x7f23e819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8113f50 con 0x7f23e819ed00 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8113dd0 con 0x7f23e8103e10 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.147+0000 7f23ed200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81140d0 con 0x7f23e8108ee0 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e819ed00 0x7f23e819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e819ed00 0x7f23e819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37682/0 (socket says 172.21.15.94:37682) 2024-06-03T01:12:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23e8103e10 0x7f23e819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 -- 172.21.15.94:0/3076943885 learned_addr learned my addr 172.21.15.94:0/3076943885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e5400640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e8108ee0 0x7f23e819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 -- 172.21.15.94:0/3076943885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e8108ee0 msgr2=0x7f23e819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e8108ee0 0x7f23e819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 -- 172.21.15.94:0/3076943885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23e8103e10 msgr2=0x7f23e819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23e8103e10 0x7f23e819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 -- 172.21.15.94:0/3076943885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e819f7f0 con 0x7f23e819ed00 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e5400640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e8108ee0 0x7f23e819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.148+0000 7f23e6800640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e819ed00 0x7f23e819f1b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009fd0 tx=0x7f23e000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.149+0000 7f23dea00640 1 -- 172.21.15.94:0/3076943885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0017070 con 0x7f23e819ed00 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.149+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e819fae0 con 0x7f23e819ed00 2024-06-03T01:12:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.149+0000 7f23dea00640 1 -- 172.21.15.94:0/3076943885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e0003e50 con 0x7f23e819ed00 2024-06-03T01:12:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.149+0000 7f23dea00640 1 -- 172.21.15.94:0/3076943885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00106c0 con 0x7f23e819ed00 2024-06-03T01:12:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.149+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81a72a0 con 0x7f23e819ed00 2024-06-03T01:12:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.152+0000 7f23dea00640 1 -- 172.21.15.94:0/3076943885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f23e0003970 con 0x7f23e819ed00 2024-06-03T01:12:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.152+0000 7f23eff6c740 1 Processor -- start 2024-06-03T01:12:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.152+0000 7f23eff6c740 1 -- start start 2024-06-03T01:12:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b94dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94eadc0 0x5621b94ed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23eff6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5621b9444420 con 0x5621b94e0040 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23eff6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5621b93ebc50 con 0x5621b94e15b0 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23eff6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5621b94403e0 con 0x5621b94eadc0 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23e5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b94dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23e5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b94dfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:49996/0 (socket says 172.21.15.94:49996) 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94e04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37688/0 (socket says 172.21.15.94:37688) 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23e5400640 1 -- 172.21.15.94:0/1284444081 learned_addr learned my addr 172.21.15.94:0/1284444081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.153+0000 7f23e5400640 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94eadc0 0x5621b94ed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23e5e00640 1 -- 172.21.15.94:0/1284444081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94eadc0 msgr2=0x5621b94ed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23e5e00640 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94eadc0 0x5621b94ed1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23e5e00640 1 -- 172.21.15.94:0/1284444081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 msgr2=0x5621b94dfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23e5e00640 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b94dfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23e5e00640 1 -- 172.21.15.94:0/1284444081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b9158210 con 0x5621b94e0040 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23dea00640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f23cc0a2da0 0x7f23cc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23dea00640 1 -- 172.21.15.94:0/3076943885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f23e00cf4f0 con 0x7f23e819ed00 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23e5e00640 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94e04c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009c20 tx=0x7f23d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc47353f7d010b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23e5400640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f23cc0a2da0 0x7f23cc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23d7400640 1 -- 172.21.15.94:0/1284444081 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004050 con 0x5621b94e0040 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23d7400640 1 -- 172.21.15.94:0/1284444081 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d8005500 con 0x5621b94e0040 2024-06-03T01:12:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.154+0000 7f23d7400640 1 -- 172.21.15.94:0/1284444081 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8004e00 con 0x5621b94e0040 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 -- 172.21.15.94:0/1284444081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 msgr2=0x5621b94e04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23e5400640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f23cc0a2da0 0x7f23cc0a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x5621b94dd7c0 tx=0x7f23d0040040 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94e04c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009c20 tx=0x7f23d8011810 comp rx=0 tx=0).stop 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 -- 172.21.15.94:0/1284444081 shutdown_connections 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94eadc0 0x5621b94ed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94e04c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 --2- 172.21.15.94:0/1284444081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b94dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 -- 172.21.15.94:0/1284444081 >> 172.21.15.94:0/1284444081 conn(0x5621b94a8f80 msgr2=0x5621b94dc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.155+0000 7f23eff6c740 1 -- 172.21.15.94:0/1284444081 shutdown_connections 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.156+0000 7f23eff6c740 1 -- 172.21.15.94:0/1284444081 wait complete. 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.156+0000 7f23eff6c740 1 -- start start 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.156+0000 7f23eff6c740 1 Processor -- start 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e15b0 0x5621b94e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23eff6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5621b94edb90 con 0x5621b94e15b0 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23eff6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5621b93ff0b0 con 0x5621b94eadc0 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23eff6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5621b94edd10 con 0x5621b94e0040 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23eff6c740 10 client.0 _ll_register_callbacks cb 0x5621b93eaf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e15b0 0x5621b94e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e15b0 0x5621b94e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37704/0 (socket says 172.21.15.94:37704) 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e5400640 1 -- 172.21.15.94:0/3109940918 learned_addr learned my addr 172.21.15.94:0/3109940918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e5e00640 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e6800640 1 -- 172.21.15.94:0/3109940918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 msgr2=0x5621b94e7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e6800640 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94e7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e6800640 1 -- 172.21.15.94:0/3109940918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e15b0 msgr2=0x5621b94e8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e6800640 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e15b0 0x5621b94e8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.157+0000 7f23e6800640 1 -- 172.21.15.94:0/3109940918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b9224620 con 0x5621b94eadc0 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23e6800640 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94e8860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23e00cc8e0 tx=0x7f23e009b2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2206cd67513ab59 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23cbe00640 1 -- 172.21.15.94:0/3109940918 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0020040 con 0x5621b94eadc0 2024-06-03T01:12:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d8005500 con 0x5621b94eadc0 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23cbe00640 1 -- 172.21.15.94:0/3109940918 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f23e0097d20 con 0x5621b94eadc0 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23cbe00640 1 -- 172.21.15.94:0/3109940918 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f23e0086070 con 0x5621b94eadc0 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23eff6c740 10 client.25457 fetch_fsmap learned FSMap version 31 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5621b94f9f50 con 0x5621b94eadc0 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.158+0000 7f23cbe00640 1 -- 172.21.15.94:0/3109940918 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d8005500 con 0x5621b94eadc0 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23cbe00640 1 -- 172.21.15.94:0/3109940918 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f23e00bb040 con 0x5621b94eadc0 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23eff6c740 10 client.25457 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23eff6c740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23eff6c740 20 client.25457 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23eff6c740 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b94fb480 0x5621b94fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23eff6c740 4 client.25457 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621b9451b50 con 0x5621b94fb480 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.159+0000 7f23e5e00640 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b94fb480 0x5621b94fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.160+0000 7f23e5e00640 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b94fb480 0x5621b94fd890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f9abc07d107585 server_cookie=e1aa3392d39a9975 in_seq=0 out_seq=0 2024-06-03T01:12:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.160+0000 7f23cbe00640 10 client.25457 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23cbe00640 1 -- 172.21.15.94:0/3109940918 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621b9451b50 con 0x5621b94fb480 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23cbe00640 10 client.25457 handle_command_reply: tid=0 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 client.25457 shutdown 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 msgr2=0x5621b94e8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94e8860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23e00cc8e0 tx=0x7f23e009b2b0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 shutdown_connections 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b94fb480 0x5621b94fd890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94e8860 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e15b0 0x5621b94e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23cbe00640 0 client.25457 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 --2- 172.21.15.94:0/3109940918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 >> 172.21.15.94:0/3109940918 conn(0x5621b94a8f80 msgr2=0x5621b94e2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 shutdown_connections 2024-06-03T01:12:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 1 -- 172.21.15.94:0/3109940918 wait complete. 2024-06-03T01:12:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.161+0000 7f23eff6c740 20 client.25457 trim_cache size 0 max 16384 2024-06-03T01:12:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 Processor -- start 2024-06-03T01:12:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 -- start start 2024-06-03T01:12:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b95031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94eadc0 0x5621b95036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b0e90 con 0x5621b94eadc0 2024-06-03T01:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b2a60 con 0x5621b94e15b0 2024-06-03T01:12:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.178+0000 7f23eff6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b1560 con 0x5621b94e0040 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94eadc0 0x5621b95036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e5400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b95031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94eadc0 0x5621b95036e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37708/0 (socket says 172.21.15.94:37708) 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 -- 172.21.15.94:0/2809047840 learned_addr learned my addr 172.21.15.94:0/2809047840 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e5e00640 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 -- 172.21.15.94:0/2809047840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 msgr2=0x5621b94dc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94dc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 -- 172.21.15.94:0/2809047840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 msgr2=0x5621b95031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b95031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e6800640 1 -- 172.21.15.94:0/2809047840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b94d7dc0 con 0x5621b94eadc0 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e5e00640 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.179+0000 7f23e5400640 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b95031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23e6800640 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94eadc0 0x5621b95036e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f23e00cc990 tx=0x7f23e009b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea274bedcb4d378 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23d6a00640 1 -- 172.21.15.94:0/2809047840 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0020070 con 0x5621b94eadc0 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23d6a00640 1 -- 172.21.15.94:0/2809047840 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e009fa60 con 0x5621b94eadc0 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23d6a00640 1 -- 172.21.15.94:0/2809047840 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e009b6b0 con 0x5621b94eadc0 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23eff6c740 1 -- 172.21.15.94:0/2809047840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94eadc0 msgr2=0x5621b95036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94eadc0 0x5621b95036e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f23e00cc990 tx=0x7f23e009b320 comp rx=0 tx=0).stop 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23eff6c740 1 -- 172.21.15.94:0/2809047840 shutdown_connections 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94eadc0 0x5621b95036e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94e15b0 0x5621b95031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2809047840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e0040 0x5621b94dc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.180+0000 7f23eff6c740 1 -- 172.21.15.94:0/2809047840 >> 172.21.15.94:0/2809047840 conn(0x5621b94a8f80 msgr2=0x5621b94dc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.181+0000 7f23eff6c740 1 -- 172.21.15.94:0/2809047840 shutdown_connections 2024-06-03T01:12:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.181+0000 7f23eff6c740 1 -- 172.21.15.94:0/2809047840 wait complete. 2024-06-03T01:12:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.181+0000 7f23eff6c740 1 -- start start 2024-06-03T01:12:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.181+0000 7f23eff6c740 1 Processor -- start 2024-06-03T01:12:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e15b0 0x5621b94ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23eff6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23eff6c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b2300 con 0x5621b94e0040 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23eff6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81b1c30 con 0x5621b94eadc0 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23eff6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81aee70 con 0x5621b94e15b0 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23eff6c740 10 client.0 _ll_register_callbacks cb 0x7f23e00092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37716/0 (socket says 172.21.15.94:37716) 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 -- 172.21.15.94:0/2886974349 learned_addr learned my addr 172.21.15.94:0/2886974349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e6800640 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5400640 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e15b0 0x5621b94ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 -- 172.21.15.94:0/2886974349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e15b0 msgr2=0x5621b94ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e15b0 0x5621b94ee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 -- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 msgr2=0x5621b94eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 -- 172.21.15.94:0/2886974349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b94ef240 con 0x5621b94e0040 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23e5e00640 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94f3450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23d8014040 tx=0x7f23d8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb644f987fe6a5b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.182+0000 7f23cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23cbe00640 1 -- 172.21.15.94:0/2886974349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d802e3c0 con 0x5621b94e0040 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23cbe00640 1 -- 172.21.15.94:0/2886974349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f23d802e560 con 0x5621b94e0040 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e009fa60 con 0x5621b94e0040 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23cbe00640 1 -- 172.21.15.94:0/2886974349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f23d803b050 con 0x5621b94e0040 2024-06-03T01:12:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23eff6c740 10 client.15339 fetch_fsmap learned FSMap version 31 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5621b9565ed0 con 0x5621b94e0040 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23cbe00640 1 -- 172.21.15.94:0/2886974349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e009fa60 con 0x5621b94e0040 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.183+0000 7f23cbe00640 1 -- 172.21.15.94:0/2886974349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 31) ==== 8374+0+0 (secure 0 0 0) 0x7f23d8036030 con 0x5621b94e0040 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23eff6c740 10 client.15339 fetch_fsmap finished waiting for FSMap version 31 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23eff6c740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23eff6c740 20 client.15339 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b9566590 0x5621b95689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23eff6c740 4 client.15339 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621b944bea0 con 0x5621b9566590 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23e5400640 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b9566590 0x5621b95689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23e5400640 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b9566590 0x5621b95689b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165c42aeaccb2c2d server_cookie=7ba1ed44193a6a5d in_seq=0 out_seq=0 2024-06-03T01:12:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.184+0000 7f23cbe00640 10 client.15339 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23cbe00640 1 -- 172.21.15.94:0/2886974349 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5621b944bea0 con 0x5621b9566590 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23cbe00640 10 client.15339 handle_command_reply: tid=0 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 client.15339 shutdown 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 msgr2=0x5621b94f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94f3450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23d8014040 tx=0x7f23d8011de0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 shutdown_connections 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5621b9566590 0x5621b95689b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5621b94eadc0 0x5621b94eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621b94e15b0 0x5621b94ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 --2- 172.21.15.94:0/2886974349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5621b94e0040 0x5621b94f3450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 >> 172.21.15.94:0/2886974349 conn(0x5621b94a8f80 msgr2=0x5621b94daf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.214+0000 7f23cbe00640 0 client.15339 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.215+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 shutdown_connections 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.215+0000 7f23eff6c740 1 -- 172.21.15.94:0/2886974349 wait complete. 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.215+0000 7f23eff6c740 20 client.15339 trim_cache size 0 max 16384 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:12:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.215+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f23cc0a2da0 msgr2=0x7f23cc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.215+0000 7f23ed200640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f23cc0a2da0 0x7f23cc0a5260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x5621b94dd7c0 tx=0x7f23d0040040 comp rx=0 tx=0).stop 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.215+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e819ed00 msgr2=0x7f23e819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.215+0000 7f23ed200640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e819ed00 0x7f23e819f1b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009fd0 tx=0x7f23e000b260 comp rx=0 tx=0).stop 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 shutdown_connections 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f23cc0a2da0 0x7f23cc0a5260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e819ed00 0x7f23e819f1b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e8108ee0 0x7f23e819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 --2- 172.21.15.94:0/3076943885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23e8103e10 0x7f23e819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 >> 172.21.15.94:0/3076943885 conn(0x7f23e80ff6d0 msgr2=0x7f23e8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 shutdown_connections 2024-06-03T01:12:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:23.216+0000 7f23ed200640 1 -- 172.21.15.94:0/3076943885 wait complete. 2024-06-03T01:12:24.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32754 6.24 MB/sec warmup 31 sec latency 2952.208 ms 2024-06-03T01:12:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:23 smithi107 ceph-mon[36622]: pgmap v64: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-06-03T01:12:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:23 smithi094 ceph-mon[32119]: pgmap v64: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-06-03T01:12:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:23 smithi179 ceph-mon[37041]: pgmap v64: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-06-03T01:12:25.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32754 6.05 MB/sec warmup 32 sec latency 3952.296 ms 2024-06-03T01:12:25.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:24 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:25.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:24 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:25.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:24 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:12:26.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32756 5.86 MB/sec warmup 33 sec latency 4943.094 ms 2024-06-03T01:12:26.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:25 smithi107 ceph-mon[36622]: pgmap v65: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 27 op/s 2024-06-03T01:12:26.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:25 smithi094 ceph-mon[32119]: pgmap v65: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 27 op/s 2024-06-03T01:12:26.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:25 smithi179 ceph-mon[37041]: pgmap v65: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 27 op/s 2024-06-03T01:12:27.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32943 5.70 MB/sec warmup 34 sec latency 836.113 ms 2024-06-03T01:12:28.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32943 5.54 MB/sec warmup 35 sec latency 1836.254 ms 2024-06-03T01:12:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:27 smithi107 ceph-mon[36622]: pgmap v66: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 32 op/s 2024-06-03T01:12:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:27 smithi094 ceph-mon[32119]: pgmap v66: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 32 op/s 2024-06-03T01:12:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:27 smithi179 ceph-mon[37041]: pgmap v66: 129 pgs: 129 active+clean; 27 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 32 op/s 2024-06-03T01:12:29.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32943 5.39 MB/sec warmup 36 sec latency 2836.388 ms 2024-06-03T01:12:30.056 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32943 5.24 MB/sec warmup 37 sec latency 3836.548 ms 2024-06-03T01:12:30.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:29 smithi107 ceph-mon[36622]: pgmap v67: 129 pgs: 129 active+clean; 27 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-06-03T01:12:30.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:29 smithi094 ceph-mon[32119]: pgmap v67: 129 pgs: 129 active+clean; 27 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-06-03T01:12:30.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:29 smithi179 ceph-mon[37041]: pgmap v67: 129 pgs: 129 active+clean; 27 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 3.8 MiB/s wr, 161 op/s 2024-06-03T01:12:31.057 INFO:tasks.workunit.client.0.smithi094.stdout: 1 32944 5.10 MB/sec warmup 38 sec latency 4829.863 ms 2024-06-03T01:12:32.057 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34884 5.27 MB/sec warmup 39 sec latency 14.581 ms 2024-06-03T01:12:32.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:31 smithi107 ceph-mon[36622]: pgmap v68: 129 pgs: 129 active+clean; 27 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 146 op/s 2024-06-03T01:12:32.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:31 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:12:32.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:31 smithi094 ceph-mon[32119]: pgmap v68: 129 pgs: 129 active+clean; 27 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 146 op/s 2024-06-03T01:12:32.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:31 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:12:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:31 smithi179 ceph-mon[37041]: pgmap v68: 129 pgs: 129 active+clean; 27 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 3.8 MiB/s wr, 146 op/s 2024-06-03T01:12:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:31 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:12:33.057 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34893 5.14 MB/sec warmup 40 sec latency 991.024 ms 2024-06-03T01:12:33.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:32 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:12:33.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:32 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:12:33.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:32 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:12:33.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:32 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:12:33.520 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:32 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:12:33.520 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:32 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:12:33.520 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:32 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:12:33.520 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:32 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:12:33.521 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:32 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:12:34.057 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34893 5.01 MB/sec warmup 41 sec latency 1991.163 ms 2024-06-03T01:12:34.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:34 smithi107 ceph-mon[36622]: pgmap v69: 129 pgs: 129 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 3.9 MiB/s wr, 164 op/s 2024-06-03T01:12:34.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:34 smithi094 ceph-mon[32119]: pgmap v69: 129 pgs: 129 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 3.9 MiB/s wr, 164 op/s 2024-06-03T01:12:34.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:34 smithi179 ceph-mon[37041]: pgmap v69: 129 pgs: 129 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 3.9 MiB/s wr, 164 op/s 2024-06-03T01:12:35.057 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34893 4.90 MB/sec warmup 42 sec latency 2991.306 ms 2024-06-03T01:12:36.057 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34986 4.78 MB/sec warmup 43 sec latency 3244.408 ms 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 -- 172.21.15.94:0/4193768082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe601129b0 msgr2=0x7efe60114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 --2- 172.21.15.94:0/4193768082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe601129b0 0x7efe60114da0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efe58009ca0 tx=0x7efe5802a950 comp rx=0 tx=0).stop 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 -- 172.21.15.94:0/4193768082 shutdown_connections 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 --2- 172.21.15.94:0/4193768082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe601129b0 0x7efe60114da0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 --2- 172.21.15.94:0/4193768082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe60074660 0x7efe60112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 --2- 172.21.15.94:0/4193768082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60073d40 0x7efe60074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 -- 172.21.15.94:0/4193768082 >> 172.21.15.94:0/4193768082 conn(0x7efe600fd430 msgr2=0x7efe600ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 -- 172.21.15.94:0/4193768082 shutdown_connections 2024-06-03T01:12:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.356+0000 7efe64c00640 1 -- 172.21.15.94:0/4193768082 wait complete. 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.357+0000 7efe64c00640 1 Processor -- start 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe64c00640 1 -- start start 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe64c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe60073d40 0x7efe601a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe64c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe60074660 0x7efe601a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe64c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601129b0 0x7efe6019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe64c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe601183a0 con 0x7efe60073d40 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe64c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe60118220 con 0x7efe601129b0 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe64c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe60118520 con 0x7efe60074660 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe5de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe60073d40 0x7efe601a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe5de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe60073d40 0x7efe601a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55950/0 (socket says 172.21.15.94:55950) 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe5de00640 1 -- 172.21.15.94:0/3267652519 learned_addr learned my addr 172.21.15.94:0/3267652519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe55400640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe60074660 0x7efe601a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.358+0000 7efe5e800640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601129b0 0x7efe6019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe5de00640 1 -- 172.21.15.94:0/3267652519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe60074660 msgr2=0x7efe601a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe5de00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe60074660 0x7efe601a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe5de00640 1 -- 172.21.15.94:0/3267652519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601129b0 msgr2=0x7efe6019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe5de00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601129b0 0x7efe6019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe5de00640 1 -- 172.21.15.94:0/3267652519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6019eda0 con 0x7efe60073d40 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe5e800640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601129b0 0x7efe6019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe5de00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe60073d40 0x7efe601a4470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe5000e9f0 tx=0x7efe5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe57400640 1 -- 172.21.15.94:0/3267652519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5000cd80 con 0x7efe60073d40 2024-06-03T01:12:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe57400640 1 -- 172.21.15.94:0/3267652519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe50010910 con 0x7efe60073d40 2024-06-03T01:12:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6019f090 con 0x7efe60073d40 2024-06-03T01:12:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe57400640 1 -- 172.21.15.94:0/3267652519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50018990 con 0x7efe60073d40 2024-06-03T01:12:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.359+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe601ab330 con 0x7efe60073d40 2024-06-03T01:12:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 Processor -- start 2024-06-03T01:12:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe57400640 1 -- 172.21.15.94:0/3267652519 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efe50010ac0 con 0x7efe60073d40 2024-06-03T01:12:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 -- start start 2024-06-03T01:12:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d05b0 0x55eeda6ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6cf040 0x55eeda6cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eeda633420 con 0x55eeda6d05b0 2024-06-03T01:12:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eeda5dac50 con 0x55eeda6cf040 2024-06-03T01:12:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.363+0000 7efe67a50740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55eeda62f3e0 con 0x55eeda6d9dc0 2024-06-03T01:12:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d05b0 0x55eeda6ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d05b0 0x55eeda6ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55962/0 (socket says 172.21.15.94:55962) 2024-06-03T01:12:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 -- 172.21.15.94:0/1544675566 learned_addr learned my addr 172.21.15.94:0/1544675566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe55400640 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5de00640 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6cf040 0x55eeda6cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 -- 172.21.15.94:0/1544675566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 msgr2=0x55eeda6dc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6dc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 -- 172.21.15.94:0/1544675566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6cf040 msgr2=0x55eeda6cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6cf040 0x55eeda6cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 -- 172.21.15.94:0/1544675566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeda347210 con 0x55eeda6d05b0 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe55400640 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6dc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5de00640 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6cf040 0x55eeda6cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe57400640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7efe440a2da0 0x7efe440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.364+0000 7efe5e800640 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d05b0 0x55eeda6ceb00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe58009ca0 tx=0x7efe58009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be4921746f9add server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.365+0000 7efe57400640 1 -- 172.21.15.94:0/3267652519 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7efe500cfbb0 con 0x7efe60073d40 2024-06-03T01:12:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.365+0000 7efe5de00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7efe440a2da0 0x7efe440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.365+0000 7efe4f000640 1 -- 172.21.15.94:0/1544675566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58034040 con 0x55eeda6d05b0 2024-06-03T01:12:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.365+0000 7efe4f000640 1 -- 172.21.15.94:0/1544675566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe58003e50 con 0x55eeda6d05b0 2024-06-03T01:12:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.365+0000 7efe4f000640 1 -- 172.21.15.94:0/1544675566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580042d0 con 0x55eeda6d05b0 2024-06-03T01:12:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 -- 172.21.15.94:0/1544675566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d05b0 msgr2=0x55eeda6ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d05b0 0x55eeda6ceb00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe58009ca0 tx=0x7efe58009a20 comp rx=0 tx=0).stop 2024-06-03T01:12:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe5de00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7efe440a2da0 0x7efe440a5260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55eeda6d1d10 tx=0x7efe50000bc0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 -- 172.21.15.94:0/1544675566 shutdown_connections 2024-06-03T01:12:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6dc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6cf040 0x55eeda6cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 --2- 172.21.15.94:0/1544675566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d05b0 0x55eeda6ceb00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 -- 172.21.15.94:0/1544675566 >> 172.21.15.94:0/1544675566 conn(0x55eeda697f80 msgr2=0x55eeda6cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.366+0000 7efe67a50740 1 -- 172.21.15.94:0/1544675566 shutdown_connections 2024-06-03T01:12:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.367+0000 7efe67a50740 1 -- 172.21.15.94:0/1544675566 wait complete. 2024-06-03T01:12:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.367+0000 7efe67a50740 1 -- start start 2024-06-03T01:12:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.367+0000 7efe67a50740 1 Processor -- start 2024-06-03T01:12:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe67a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6cf040 0x55eeda6d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe67a50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe67a50740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe67a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eeda6dcb30 con 0x55eeda6cf040 2024-06-03T01:12:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe67a50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eeda5ee0b0 con 0x55eeda6d05b0 2024-06-03T01:12:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe67a50740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55eeda6dccb0 con 0x55eeda6d9dc0 2024-06-03T01:12:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe67a50740 10 client.0 _ll_register_callbacks cb 0x55eeda5d9f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6cf040 0x55eeda6d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6cf040 0x55eeda6d6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55978/0 (socket says 172.21.15.94:55978) 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 -- 172.21.15.94:0/334472314 learned_addr learned my addr 172.21.15.94:0/334472314 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe55400640 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 -- 172.21.15.94:0/334472314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 msgr2=0x55eeda6d78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 -- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 msgr2=0x55eeda6d73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.368+0000 7efe5e800640 1 -- 172.21.15.94:0/334472314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeda413620 con 0x55eeda6cf040 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe5e800640 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6cf040 0x55eeda6d6e60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efe58005cc0 tx=0x7efe5802fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a95500b3726e30 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe43400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe43400640 1 -- 172.21.15.94:0/334472314 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58055060 con 0x55eeda6cf040 2024-06-03T01:12:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe43400640 1 -- 172.21.15.94:0/334472314 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7efe58003a30 con 0x55eeda6cf040 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe58003e50 con 0x55eeda6cf040 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe43400640 1 -- 172.21.15.94:0/334472314 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe58034070 con 0x55eeda6cf040 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe67a50740 10 client.15351 fetch_fsmap learned FSMap version 32 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.369+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eeda6dcb30 con 0x55eeda6cf040 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe43400640 1 -- 172.21.15.94:0/334472314 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe58003e50 con 0x55eeda6cf040 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe43400640 1 -- 172.21.15.94:0/334472314 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7efe58042070 con 0x55eeda6cf040 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe67a50740 10 client.15351 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe67a50740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe67a50740 20 client.15351 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe67a50740 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda6ea0d0 0x55eeda6ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe67a50740 4 client.15351 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:12:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeda640b50 con 0x55eeda6ea0d0 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.370+0000 7efe55400640 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda6ea0d0 0x55eeda6ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.371+0000 7efe55400640 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda6ea0d0 0x55eeda6ec4c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c11daae604814 server_cookie=f2d9af9217c84171 in_seq=0 out_seq=0 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.371+0000 7efe43400640 10 client.15351 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe43400640 1 -- 172.21.15.94:0/334472314 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eeda640b50 con 0x55eeda6ea0d0 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe43400640 10 client.15351 handle_command_reply: tid=0 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 client.15351 shutdown 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6cf040 msgr2=0x55eeda6d6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6cf040 0x55eeda6d6e60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efe58005cc0 tx=0x7efe5802fea0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 shutdown_connections 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda6ea0d0 0x55eeda6ec4c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 --2- 172.21.15.94:0/334472314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6cf040 0x55eeda6d6e60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe43400640 0 client.15351 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 >> 172.21.15.94:0/334472314 conn(0x55eeda697f80 msgr2=0x55eeda6d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 shutdown_connections 2024-06-03T01:12:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 1 -- 172.21.15.94:0/334472314 wait complete. 2024-06-03T01:12:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.372+0000 7efe67a50740 20 client.15351 trim_cache size 0 max 16384 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.389+0000 7efe67a50740 1 Processor -- start 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.389+0000 7efe67a50740 1 -- start start 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.389+0000 7efe67a50740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe67a50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe67a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe67a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe601b53f0 con 0x55eeda6d9dc0 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe67a50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe601b6ff0 con 0x55eeda6d05b0 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe67a50740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe601b5ac0 con 0x55eeda6cf040 2024-06-03T01:12:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55992/0 (socket says 172.21.15.94:55992) 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 -- 172.21.15.94:0/2162116212 learned_addr learned my addr 172.21.15.94:0/2162116212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe55400640 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 -- 172.21.15.94:0/2162116212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 msgr2=0x55eeda6d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 -- 172.21.15.94:0/2162116212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 msgr2=0x55eeda6d7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe5e800640 1 -- 172.21.15.94:0/2162116212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeda6cfb10 con 0x55eeda6d9dc0 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.390+0000 7efe55400640 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.391+0000 7efe5e800640 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6caf90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efe58009620 tx=0x7efe5802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdd8d6ced6b90b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.391+0000 7efe4e600640 1 -- 172.21.15.94:0/2162116212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58055040 con 0x55eeda6d9dc0 2024-06-03T01:12:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.391+0000 7efe4e600640 1 -- 172.21.15.94:0/2162116212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe58050820 con 0x55eeda6d9dc0 2024-06-03T01:12:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.391+0000 7efe4e600640 1 -- 172.21.15.94:0/2162116212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580045a0 con 0x55eeda6d9dc0 2024-06-03T01:12:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 -- 172.21.15.94:0/2162116212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 msgr2=0x55eeda6caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6caf90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efe58009620 tx=0x7efe5802a950 comp rx=0 tx=0).stop 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 -- 172.21.15.94:0/2162116212 shutdown_connections 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6caf90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 --2- 172.21.15.94:0/2162116212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 -- 172.21.15.94:0/2162116212 >> 172.21.15.94:0/2162116212 conn(0x55eeda697f80 msgr2=0x55eeda6d6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 -- 172.21.15.94:0/2162116212 shutdown_connections 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 -- 172.21.15.94:0/2162116212 wait complete. 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 -- start start 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.392+0000 7efe67a50740 1 Processor -- start 2024-06-03T01:12:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe67a50740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe67a50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe67a50740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe67a50740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efe601b6860 con 0x55eeda6d9dc0 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe67a50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe601b6190 con 0x55eeda6d05b0 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe67a50740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe601b33d0 con 0x55eeda6cf040 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe67a50740 10 client.0 _ll_register_callbacks cb 0x7efe48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe5e800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe55400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe5e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe55400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37426/0 (socket says 172.21.15.94:37426) 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe5e800640 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56000/0 (socket says 172.21.15.94:56000) 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.393+0000 7efe55400640 1 -- 172.21.15.94:0/1411508656 learned_addr learned my addr 172.21.15.94:0/1411508656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe55400640 1 -- 172.21.15.94:0/1411508656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 msgr2=0x55eeda6d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe55400640 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe55400640 1 -- 172.21.15.94:0/1411508656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 msgr2=0x55eeda6d4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe55400640 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe55400640 1 -- 172.21.15.94:0/1411508656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeda753e20 con 0x55eeda6d05b0 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe5e800640 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe55400640 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d4590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efe48004700 tx=0x7efe48004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbc5f70694a861f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe43400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe43400640 1 -- 172.21.15.94:0/1411508656 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4801f400 con 0x55eeda6d05b0 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe58050820 con 0x55eeda6d05b0 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.394+0000 7efe43400640 1 -- 172.21.15.94:0/1411508656 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7efe4801f5a0 con 0x55eeda6d05b0 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe43400640 1 -- 172.21.15.94:0/1411508656 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efe4801f7a0 con 0x55eeda6d05b0 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 10 client.25481 fetch_fsmap learned FSMap version 32 2024-06-03T01:12:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55eeda755150 con 0x55eeda6d05b0 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe43400640 1 -- 172.21.15.94:0/1411508656 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe58050820 con 0x55eeda6d05b0 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe43400640 1 -- 172.21.15.94:0/1411508656 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7efe48027bc0 con 0x55eeda6d05b0 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 10 client.25481 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 10 client.25481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 20 client.25481 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda755960 0x55eeda757d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 4 client.25481 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:12:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eeda63aea0 con 0x55eeda755960 2024-06-03T01:12:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.395+0000 7efe5e800640 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda755960 0x55eeda757d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.396+0000 7efe5e800640 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda755960 0x55eeda757d80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7aaf7a6d442174 server_cookie=694be044c3b3958d in_seq=0 out_seq=0 2024-06-03T01:12:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.396+0000 7efe43400640 10 client.25481 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe43400640 1 -- 172.21.15.94:0/1411508656 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eeda63aea0 con 0x55eeda755960 2024-06-03T01:12:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe43400640 10 client.25481 handle_command_reply: tid=0 2024-06-03T01:12:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 client.25481 shutdown 2024-06-03T01:12:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 msgr2=0x55eeda6d4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d4590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efe48004700 tx=0x7efe48004570 comp rx=0 tx=0).stop 2024-06-03T01:12:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 shutdown_connections 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeda6d9dc0 0x55eeda6d4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeda755960 0x55eeda757d80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeda6d05b0 0x55eeda6d4590 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe43400640 0 client.25481 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 --2- 172.21.15.94:0/1411508656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeda6cf040 0x55eeda6d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.430+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 >> 172.21.15.94:0/1411508656 conn(0x55eeda697f80 msgr2=0x55eeda6d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.431+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 shutdown_connections 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.431+0000 7efe67a50740 1 -- 172.21.15.94:0/1411508656 wait complete. 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.431+0000 7efe67a50740 20 client.25481 trim_cache size 0 max 16384 2024-06-03T01:12:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:12:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:12:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:12:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.431+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7efe440a2da0 msgr2=0x7efe440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.431+0000 7efe64c00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7efe440a2da0 0x7efe440a5260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55eeda6d1d10 tx=0x7efe50000bc0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.431+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe60073d40 msgr2=0x7efe601a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.431+0000 7efe64c00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe60073d40 0x7efe601a4470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe5000e9f0 tx=0x7efe5000eeb0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 shutdown_connections 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7efe440a2da0 0x7efe440a5260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601129b0 0x7efe6019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe60074660 0x7efe601a49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 --2- 172.21.15.94:0/3267652519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe60073d40 0x7efe601a4470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 >> 172.21.15.94:0/3267652519 conn(0x7efe600fd430 msgr2=0x7efe600fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 shutdown_connections 2024-06-03T01:12:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:36.432+0000 7efe64c00640 1 -- 172.21.15.94:0/3267652519 wait complete. 2024-06-03T01:12:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:36 smithi107 ceph-mon[36622]: pgmap v70: 129 pgs: 129 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-06-03T01:12:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:36 smithi094 ceph-mon[32119]: pgmap v70: 129 pgs: 129 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-06-03T01:12:36.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:36 smithi179 ceph-mon[37041]: pgmap v70: 129 pgs: 129 active+clean; 27 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 2.5 MiB/s wr, 151 op/s 2024-06-03T01:12:37.057 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34986 4.67 MB/sec warmup 44 sec latency 1645.139 ms 2024-06-03T01:12:38.058 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34986 4.57 MB/sec warmup 45 sec latency 2645.279 ms 2024-06-03T01:12:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:38 smithi107 ceph-mon[36622]: pgmap v71: 129 pgs: 129 active+clean; 27 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.5 MiB/s wr, 153 op/s 2024-06-03T01:12:38.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:38 smithi094 ceph-mon[32119]: pgmap v71: 129 pgs: 129 active+clean; 27 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.5 MiB/s wr, 153 op/s 2024-06-03T01:12:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:38 smithi179 ceph-mon[37041]: pgmap v71: 129 pgs: 129 active+clean; 27 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 2.5 MiB/s wr, 153 op/s 2024-06-03T01:12:39.058 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34986 4.47 MB/sec warmup 46 sec latency 3645.460 ms 2024-06-03T01:12:40.058 INFO:tasks.workunit.client.0.smithi094.stdout: 1 34986 4.38 MB/sec warmup 47 sec latency 4645.560 ms 2024-06-03T01:12:40.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:40 smithi107 ceph-mon[36622]: pgmap v72: 129 pgs: 129 active+clean; 24 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-06-03T01:12:40.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:40 smithi094 ceph-mon[32119]: pgmap v72: 129 pgs: 129 active+clean; 24 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-06-03T01:12:40.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:40 smithi179 ceph-mon[37041]: pgmap v72: 129 pgs: 129 active+clean; 24 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-06-03T01:12:41.058 INFO:tasks.workunit.client.0.smithi094.stdout: 1 35138 4.30 MB/sec warmup 48 sec latency 4892.147 ms 2024-06-03T01:12:42.058 INFO:tasks.workunit.client.0.smithi094.stdout: 1 35138 4.21 MB/sec warmup 49 sec latency 1727.729 ms 2024-06-03T01:12:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:42 smithi107 ceph-mon[36622]: pgmap v73: 129 pgs: 129 active+clean; 24 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 41 op/s 2024-06-03T01:12:42.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:42 smithi094 ceph-mon[32119]: pgmap v73: 129 pgs: 129 active+clean; 24 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 41 op/s 2024-06-03T01:12:42.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:42 smithi179 ceph-mon[37041]: pgmap v73: 129 pgs: 129 active+clean; 24 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 112 KiB/s wr, 41 op/s 2024-06-03T01:12:43.058 INFO:tasks.workunit.client.0.smithi094.stdout: 1 35138 4.12 MB/sec warmup 50 sec latency 2727.906 ms 2024-06-03T01:12:44.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 35138 4.04 MB/sec warmup 51 sec latency 3728.067 ms 2024-06-03T01:12:44.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:44 smithi107 ceph-mon[36622]: pgmap v74: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 48 op/s 2024-06-03T01:12:44.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:44 smithi094 ceph-mon[32119]: pgmap v74: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 48 op/s 2024-06-03T01:12:44.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:44 smithi179 ceph-mon[37041]: pgmap v74: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 120 KiB/s wr, 48 op/s 2024-06-03T01:12:45.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 35138 3.97 MB/sec warmup 52 sec latency 4728.231 ms 2024-06-03T01:12:46.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 35879 4.00 MB/sec warmup 53 sec latency 4975.502 ms 2024-06-03T01:12:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:46 smithi107 ceph-mon[36622]: pgmap v75: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-06-03T01:12:46.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:46 smithi094 ceph-mon[32119]: pgmap v75: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-06-03T01:12:46.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:46 smithi179 ceph-mon[37041]: pgmap v75: 129 pgs: 129 active+clean; 24 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-06-03T01:12:47.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36381 3.94 MB/sec warmup 54 sec latency 264.609 ms 2024-06-03T01:12:48.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36381 3.87 MB/sec warmup 55 sec latency 1264.753 ms 2024-06-03T01:12:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:48 smithi107 ceph-mon[36622]: pgmap v76: 129 pgs: 129 active+clean; 24 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 66 KiB/s wr, 34 op/s 2024-06-03T01:12:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:48 smithi094 ceph-mon[32119]: pgmap v76: 129 pgs: 129 active+clean; 24 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 66 KiB/s wr, 34 op/s 2024-06-03T01:12:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:48 smithi179 ceph-mon[37041]: pgmap v76: 129 pgs: 129 active+clean; 24 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 66 KiB/s wr, 34 op/s 2024-06-03T01:12:49.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36381 3.80 MB/sec warmup 56 sec latency 2264.894 ms 2024-06-03T01:12:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 -- 172.21.15.94:0/2943862723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 msgr2=0x7f920c10e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 --2- 172.21.15.94:0/2943862723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 0x7f920c10e250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009930 tx=0x7f91fc02a420 comp rx=0 tx=0).stop 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 -- 172.21.15.94:0/2943862723 shutdown_connections 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 --2- 172.21.15.94:0/2943862723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 0x7f920c114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 --2- 172.21.15.94:0/2943862723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f920c107ee0 0x7f920c108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 --2- 172.21.15.94:0/2943862723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 0x7f920c10e250 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 -- 172.21.15.94:0/2943862723 >> 172.21.15.94:0/2943862723 conn(0x7f920c077780 msgr2=0x7f920c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 -- 172.21.15.94:0/2943862723 shutdown_connections 2024-06-03T01:12:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.573+0000 7f9212600640 1 -- 172.21.15.94:0/2943862723 wait complete. 2024-06-03T01:12:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.574+0000 7f9212600640 1 Processor -- start 2024-06-03T01:12:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.574+0000 7f9212600640 1 -- start start 2024-06-03T01:12:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9212600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f920c107ee0 0x7f920c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9212600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 0x7f920c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9212600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 0x7f920c1a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9212600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f920c118170 con 0x7f920c107ee0 2024-06-03T01:12:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9212600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f920c117ff0 con 0x7f920c108860 2024-06-03T01:12:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9212600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1182f0 con 0x7f920c10de70 2024-06-03T01:12:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 0x7f920c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9211200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f920c107ee0 0x7f920c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 0x7f920c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45908/0 (socket says 172.21.15.94:45908) 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 -- 172.21.15.94:0/3554244878 learned_addr learned my addr 172.21.15.94:0/3554244878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f9211c00640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 0x7f920c1a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 -- 172.21.15.94:0/3554244878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 msgr2=0x7f920c1a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 0x7f920c1a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 -- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f920c107ee0 msgr2=0x7f920c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f920c107ee0 0x7f920c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.575+0000 7f920be00640 1 -- 172.21.15.94:0/3554244878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c1a36a0 con 0x7f920c108860 2024-06-03T01:12:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.576+0000 7f920be00640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 0x7f920c19ebf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91f800ebc0 tx=0x7f91f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.576+0000 7f9209600640 1 -- 172.21.15.94:0/3554244878 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8010630 con 0x7f920c108860 2024-06-03T01:12:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.576+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c1a3990 con 0x7f920c108860 2024-06-03T01:12:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.576+0000 7f9209600640 1 -- 172.21.15.94:0/3554244878 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8010cb0 con 0x7f920c108860 2024-06-03T01:12:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.576+0000 7f9209600640 1 -- 172.21.15.94:0/3554244878 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8018bd0 con 0x7f920c108860 2024-06-03T01:12:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.576+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c1ab220 con 0x7f920c108860 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 Processor -- start 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 -- start start 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9209600640 1 -- 172.21.15.94:0/3554244878 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f91f80107d0 con 0x7f920c108860 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a2203fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a2203fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a220406f80 0x55a220409370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a220360420 con 0x55a2203fc200 2024-06-03T01:12:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a220307c50 con 0x55a220406f80 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9216a3d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a22035c3e0 con 0x55a2203fd770 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.580+0000 7f9211200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a2203fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a2203fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55410/0 (socket says 172.21.15.94:55410) 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 -- 172.21.15.94:0/3192340695 learned_addr learned my addr 172.21.15.94:0/3192340695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f920be00640 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a220406f80 0x55a220409370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211c00640 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a2203fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 -- 172.21.15.94:0/3192340695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 msgr2=0x55a2203fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a2203fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 -- 172.21.15.94:0/3192340695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a220406f80 msgr2=0x55a220409370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a220406f80 0x55a220409370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 -- 172.21.15.94:0/3192340695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a220074210 con 0x55a2203fc200 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.581+0000 7f9211200640 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a2203fc680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91fc02a940 tx=0x7f91fc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf96fb9a44682f96 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9209600640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91f00a2cf0 0x7f91f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9202000640 1 -- 172.21.15.94:0/3192340695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc040070 con 0x55a2203fc200 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9202000640 1 -- 172.21.15.94:0/3192340695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc031870 con 0x55a2203fc200 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9202000640 1 -- 172.21.15.94:0/3192340695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc035040 con 0x55a2203fc200 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9209600640 1 -- 172.21.15.94:0/3554244878 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f91f80cfe30 con 0x7f920c108860 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9211c00640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91f00a2cf0 0x7f91f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9216a3d740 1 -- 172.21.15.94:0/3192340695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 msgr2=0x55a2203fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.582+0000 7f9216a3d740 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a2203fc680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91fc02a940 tx=0x7f91fc005bd0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 -- 172.21.15.94:0/3192340695 shutdown_connections 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a220406f80 0x55a220409370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a2203fc680 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 --2- 172.21.15.94:0/3192340695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a2203fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 -- 172.21.15.94:0/3192340695 >> 172.21.15.94:0/3192340695 conn(0x55a2203c4f80 msgr2=0x55a2203f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 -- 172.21.15.94:0/3192340695 shutdown_connections 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9211c00640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91f00a2cf0 0x7f91f00a51b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9204009d20 tx=0x7f9204005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 -- 172.21.15.94:0/3192340695 wait complete. 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 -- start start 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.583+0000 7f9216a3d740 1 Processor -- start 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a220403f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fd770 0x55a220404480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a220406f80 0x55a2204049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9216a3d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a220409cf0 con 0x55a2203fc200 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9216a3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a22031b0b0 con 0x55a2203fd770 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9216a3d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a220409e70 con 0x55a220406f80 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9216a3d740 10 client.0 _ll_register_callbacks cb 0x55a220306f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9211200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a220403f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9211200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a220403f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55418/0 (socket says 172.21.15.94:55418) 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9211200640 1 -- 172.21.15.94:0/193558617 learned_addr learned my addr 172.21.15.94:0/193558617 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f920be00640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fd770 0x55a220404480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.584+0000 7f9211c00640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a220406f80 0x55a2204049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f920be00640 1 -- 172.21.15.94:0/193558617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a220406f80 msgr2=0x55a2204049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f920be00640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a220406f80 0x55a2204049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f920be00640 1 -- 172.21.15.94:0/193558617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 msgr2=0x55a220403f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f920be00640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a220403f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f920be00640 1 -- 172.21.15.94:0/193558617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a220140620 con 0x55a2203fd770 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f9211200640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a220403f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f920be00640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fd770 0x55a220404480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f80cd190 tx=0x7f91f800eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864fcc98a8c582e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f91f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f91f6a00640 1 -- 172.21.15.94:0/193558617 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8010e60 con 0x55a2203fd770 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91fc031870 con 0x55a2203fd770 2024-06-03T01:12:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f91f6a00640 1 -- 172.21.15.94:0/193558617 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f91f80a4770 con 0x55a2203fd770 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f91f6a00640 1 -- 172.21.15.94:0/193558617 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91f809be10 con 0x55a2203fd770 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f9216a3d740 10 client.25493 fetch_fsmap learned FSMap version 32 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.585+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a220416060 con 0x55a2203fd770 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f91f6a00640 1 -- 172.21.15.94:0/193558617 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc031870 con 0x55a2203fd770 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f91f6a00640 1 -- 172.21.15.94:0/193558617 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f91f809b490 con 0x55a2203fd770 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f9216a3d740 10 client.25493 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f9216a3d740 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f9216a3d740 20 client.25493 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f9216a3d740 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220417540 0x55a2204199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f9216a3d740 4 client.25493 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:12:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a22036db50 con 0x55a220417540 2024-06-03T01:12:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.586+0000 7f9211200640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220417540 0x55a2204199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.587+0000 7f9211200640 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220417540 0x55a2204199d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=456b8c8a1a028bb2 server_cookie=3e51d10e61a19cea in_seq=0 out_seq=0 2024-06-03T01:12:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.587+0000 7f91f6a00640 10 client.25493 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f91f6a00640 1 -- 172.21.15.94:0/193558617 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a22036db50 con 0x55a220417540 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f91f6a00640 10 client.25493 handle_command_reply: tid=0 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 client.25493 shutdown 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fd770 msgr2=0x55a220404480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fd770 0x55a220404480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f80cd190 tx=0x7f91f800eec0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 shutdown_connections 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a220406f80 0x55a2204049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220417540 0x55a2204199d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fd770 0x55a220404480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 --2- 172.21.15.94:0/193558617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2203fc200 0x55a220403f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 >> 172.21.15.94:0/193558617 conn(0x55a2203c4f80 msgr2=0x55a2203fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f91f6a00640 0 client.25493 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 shutdown_connections 2024-06-03T01:12:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 1 -- 172.21.15.94:0/193558617 wait complete. 2024-06-03T01:12:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.588+0000 7f9216a3d740 20 client.25493 trim_cache size 0 max 16384 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.604+0000 7f9216a3d740 1 Processor -- start 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9216a3d740 1 -- start start 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a220402f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22041f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22041f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9216a3d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1b5230 con 0x55a220406f80 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9216a3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1b6e30 con 0x55a2203fc200 2024-06-03T01:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9216a3d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1b5900 con 0x55a2203fd770 2024-06-03T01:12:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9211200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a220402f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9211c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22041f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9211200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a220402f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45928/0 (socket says 172.21.15.94:45928) 2024-06-03T01:12:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f9211200640 1 -- 172.21.15.94:0/1397366316 learned_addr learned my addr 172.21.15.94:0/1397366316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.605+0000 7f920be00640 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22041f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9211200640 1 -- 172.21.15.94:0/1397366316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 msgr2=0x55a22041f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9211200640 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22041f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9211200640 1 -- 172.21.15.94:0/1397366316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 msgr2=0x55a22041f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9211200640 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22041f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9211200640 1 -- 172.21.15.94:0/1397366316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a220419fb0 con 0x55a2203fc200 2024-06-03T01:12:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9211200640 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a220402f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009a20 tx=0x7f91fc0054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1902d92a2ff3e118 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9201600640 1 -- 172.21.15.94:0/1397366316 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc053070 con 0x55a2203fc200 2024-06-03T01:12:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9201600640 1 -- 172.21.15.94:0/1397366316 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc031870 con 0x55a2203fc200 2024-06-03T01:12:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.606+0000 7f9201600640 1 -- 172.21.15.94:0/1397366316 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc035040 con 0x55a2203fc200 2024-06-03T01:12:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 -- 172.21.15.94:0/1397366316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 msgr2=0x55a220402f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a220402f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009a20 tx=0x7f91fc0054e0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 -- 172.21.15.94:0/1397366316 shutdown_connections 2024-06-03T01:12:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22041f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22041f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 --2- 172.21.15.94:0/1397366316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a220402f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 -- 172.21.15.94:0/1397366316 >> 172.21.15.94:0/1397366316 conn(0x55a2203c4f80 msgr2=0x55a220402810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 -- 172.21.15.94:0/1397366316 shutdown_connections 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 -- 172.21.15.94:0/1397366316 wait complete. 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.607+0000 7f9216a3d740 1 -- start start 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 1 Processor -- start 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a22040a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22040a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22040ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1b66a0 con 0x55a220406f80 2024-06-03T01:12:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1b5fd0 con 0x55a2203fc200 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1b3210 con 0x55a2203fd770 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9216a3d740 10 client.0 _ll_register_callbacks cb 0x7f91f800eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9211c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22040ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9211200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a22040a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f920be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22040a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9211200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a22040a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45936/0 (socket says 172.21.15.94:45936) 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9211200640 1 -- 172.21.15.94:0/793278227 learned_addr learned my addr 172.21.15.94:0/793278227 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.608+0000 7f9211c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22040ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55442/0 (socket says 172.21.15.94:55442) 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9211200640 1 -- 172.21.15.94:0/793278227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 msgr2=0x55a22040a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9211200640 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22040a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9211200640 1 -- 172.21.15.94:0/793278227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 msgr2=0x55a22040ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:12:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9211200640 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22040ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9211200640 1 -- 172.21.15.94:0/793278227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2204804a0 con 0x55a2203fc200 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9211c00640 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22040ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9211200640 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a22040a100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91fc005550 tx=0x7f91fc005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a43dfb4131a50b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f91f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f91f6a00640 1 -- 172.21.15.94:0/793278227 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc035070 con 0x55a2203fc200 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91fc031870 con 0x55a2203fc200 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.609+0000 7f91f6a00640 1 -- 172.21.15.94:0/793278227 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f91fc0055f0 con 0x55a2203fc200 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f91f6a00640 1 -- 172.21.15.94:0/793278227 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f91fc053040 con 0x55a2203fc200 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 10 client.25499 fetch_fsmap learned FSMap version 32 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a220481790 con 0x55a2203fc200 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f91f6a00640 1 -- 172.21.15.94:0/793278227 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc031870 con 0x55a2203fc200 2024-06-03T01:12:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f91f6a00640 1 -- 172.21.15.94:0/793278227 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f91fc05f030 con 0x55a2203fc200 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 10 client.25499 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 10 client.25499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 20 client.25499 populate_metadata read hostname 'smithi094' 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220481b90 0x55a220483f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 4 client.25499 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a220367ea0 con 0x55a220481b90 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.610+0000 7f920be00640 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220481b90 0x55a220483f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.611+0000 7f920be00640 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220481b90 0x55a220483f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba557ec99cc1cb5 server_cookie=2c3b089d7e79c6b2 in_seq=0 out_seq=0 2024-06-03T01:12:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.611+0000 7f91f6a00640 10 client.25499 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f91f6a00640 1 -- 172.21.15.94:0/793278227 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a220367ea0 con 0x55a220481b90 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f91f6a00640 10 client.25499 handle_command_reply: tid=0 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 client.25499 shutdown 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 msgr2=0x55a22040a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a22040a100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91fc005550 tx=0x7f91fc005b70 comp rx=0 tx=0).stop 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 shutdown_connections 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a220406f80 0x55a22040ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a2203fd770 0x55a22040a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a220481b90 0x55a220483f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f9216a3d740 1 --2- 172.21.15.94:0/793278227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2203fc200 0x55a22040a100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.632+0000 7f91f6a00640 0 client.25499 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:12:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.633+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 >> 172.21.15.94:0/793278227 conn(0x55a2203c4f80 msgr2=0x55a220403f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.633+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 shutdown_connections 2024-06-03T01:12:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.633+0000 7f9216a3d740 1 -- 172.21.15.94:0/793278227 wait complete. 2024-06-03T01:12:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.633+0000 7f9216a3d740 20 client.25499 trim_cache size 0 max 16384 2024-06-03T01:12:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:12:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:12:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91f00a2cf0 msgr2=0x7f91f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91f00a2cf0 0x7f91f00a51b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9204009d20 tx=0x7f9204005f90 comp rx=0 tx=0).stop 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 msgr2=0x7f920c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 0x7f920c19ebf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91f800ebc0 tx=0x7f91f800c6a0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 shutdown_connections 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91f00a2cf0 0x7f91f00a51b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f920c10de70 0x7f920c1a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f920c108860 0x7f920c19ebf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 --2- 172.21.15.94:0/3554244878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f920c107ee0 0x7f920c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:12:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 >> 172.21.15.94:0/3554244878 conn(0x7f920c077780 msgr2=0x7f920c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:12:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 shutdown_connections 2024-06-03T01:12:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:12:49.634+0000 7f9212600640 1 -- 172.21.15.94:0/3554244878 wait complete. 2024-06-03T01:12:50.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36381 3.73 MB/sec warmup 57 sec latency 3265.034 ms 2024-06-03T01:12:50.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:50 smithi107 ceph-mon[36622]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 146 KiB/s wr, 52 op/s 2024-06-03T01:12:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:50 smithi094 ceph-mon[32119]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 146 KiB/s wr, 52 op/s 2024-06-03T01:12:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:50 smithi179 ceph-mon[37041]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 146 KiB/s wr, 52 op/s 2024-06-03T01:12:51.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36394 3.67 MB/sec warmup 58 sec latency 4250.300 ms 2024-06-03T01:12:52.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36488 3.61 MB/sec warmup 59 sec latency 892.608 ms 2024-06-03T01:12:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:52 smithi107 ceph-mon[36622]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 103 KiB/s wr, 30 op/s 2024-06-03T01:12:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:52 smithi094 ceph-mon[32119]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 103 KiB/s wr, 30 op/s 2024-06-03T01:12:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:52 smithi179 ceph-mon[37041]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 103 KiB/s wr, 30 op/s 2024-06-03T01:12:53.059 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36488 3.55 MB/sec warmup 60 sec latency 1892.763 ms 2024-06-03T01:12:54.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36488 3.49 MB/sec warmup 61 sec latency 2892.909 ms 2024-06-03T01:12:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:54 smithi107 ceph-mon[36622]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 142 KiB/s wr, 46 op/s 2024-06-03T01:12:54.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:54 smithi094 ceph-mon[32119]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 142 KiB/s wr, 46 op/s 2024-06-03T01:12:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:54 smithi179 ceph-mon[37041]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 142 KiB/s wr, 46 op/s 2024-06-03T01:12:55.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36488 3.44 MB/sec warmup 62 sec latency 3893.075 ms 2024-06-03T01:12:56.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 36501 3.38 MB/sec warmup 63 sec latency 4876.978 ms 2024-06-03T01:12:56.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:56 smithi094 ceph-mon[32119]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 134 KiB/s wr, 39 op/s 2024-06-03T01:12:56.854 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:56 smithi179 ceph-mon[37041]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 134 KiB/s wr, 39 op/s 2024-06-03T01:12:57.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:56 smithi107 ceph-mon[36622]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 134 KiB/s wr, 39 op/s 2024-06-03T01:12:57.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 38257 3.49 MB/sec warmup 64 sec latency 14.232 ms 2024-06-03T01:12:58.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 39418 3.56 MB/sec warmup 65 sec latency 13.794 ms 2024-06-03T01:12:58.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:58 smithi094 ceph-mon[32119]: pgmap v81: 129 pgs: 129 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 334 KiB/s wr, 56 op/s 2024-06-03T01:12:58.854 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:58 smithi179 ceph-mon[37041]: pgmap v81: 129 pgs: 129 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 334 KiB/s wr, 56 op/s 2024-06-03T01:12:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:58 smithi107 ceph-mon[36622]: pgmap v81: 129 pgs: 129 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 334 KiB/s wr, 56 op/s 2024-06-03T01:12:59.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 39722 3.51 MB/sec warmup 66 sec latency 478.623 ms 2024-06-03T01:12:59.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:12:59 smithi094 ceph-mon[32119]: pgmap v82: 129 pgs: 129 active+clean; 26 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 510 KiB/s wr, 68 op/s 2024-06-03T01:12:59.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:12:59 smithi179 ceph-mon[37041]: pgmap v82: 129 pgs: 129 active+clean; 26 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 510 KiB/s wr, 68 op/s 2024-06-03T01:13:00.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:12:59 smithi107 ceph-mon[36622]: pgmap v82: 129 pgs: 129 active+clean; 26 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 510 KiB/s wr, 68 op/s 2024-06-03T01:13:00.060 INFO:tasks.workunit.client.0.smithi094.stdout: 1 39722 3.45 MB/sec warmup 67 sec latency 1478.813 ms 2024-06-03T01:13:01.061 INFO:tasks.workunit.client.0.smithi094.stdout: 1 39766 3.41 MB/sec warmup 68 sec latency 2437.126 ms 2024-06-03T01:13:02.061 INFO:tasks.workunit.client.0.smithi094.stdout: 1 40085 3.37 MB/sec warmup 69 sec latency 78.007 ms 2024-06-03T01:13:02.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:01 smithi094 ceph-mon[32119]: pgmap v83: 129 pgs: 129 active+clean; 26 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-06-03T01:13:02.104 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:01 smithi179 ceph-mon[37041]: pgmap v83: 129 pgs: 129 active+clean; 26 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-06-03T01:13:02.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:01 smithi107 ceph-mon[36622]: pgmap v83: 129 pgs: 129 active+clean; 26 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-06-03T01:13:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 -- 172.21.15.94:0/3565824512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400102170 msgr2=0x7f440010e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 --2- 172.21.15.94:0/3565824512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400102170 0x7f440010e680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009ca0 tx=0x7f43f802a8c0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 -- 172.21.15.94:0/3565824512 shutdown_connections 2024-06-03T01:13:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 --2- 172.21.15.94:0/3565824512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4400102170 0x7f440010e680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 --2- 172.21.15.94:0/3565824512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44001017b0 0x7f4400101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 --2- 172.21.15.94:0/3565824512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001077c0 0x7f4400107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 -- 172.21.15.94:0/3565824512 >> 172.21.15.94:0/3565824512 conn(0x7f44000fd2e0 msgr2=0x7f44000ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 -- 172.21.15.94:0/3565824512 shutdown_connections 2024-06-03T01:13:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.785+0000 7f43ff200640 1 -- 172.21.15.94:0/3565824512 wait complete. 2024-06-03T01:13:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.786+0000 7f43ff200640 1 Processor -- start 2024-06-03T01:13:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.786+0000 7f43ff200640 1 -- start start 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43ff200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001017b0 0x7f4400073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43ff200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4400102170 0x7f44000741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43ff200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44001077c0 0x7f4400074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43ff200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4400111bd0 con 0x7f4400102170 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43ff200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4400111a50 con 0x7f44001077c0 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43ff200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4400111d50 con 0x7f44001017b0 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001017b0 0x7f4400073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001017b0 0x7f4400073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49286/0 (socket says 172.21.15.94:49286) 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43fd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4400102170 0x7f44000741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43fde00640 1 -- 172.21.15.94:0/2177314691 learned_addr learned my addr 172.21.15.94:0/2177314691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.787+0000 7f43fe800640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44001077c0 0x7f4400074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43fd400640 1 -- 172.21.15.94:0/2177314691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001017b0 msgr2=0x7f4400073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43fd400640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001017b0 0x7f4400073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43fd400640 1 -- 172.21.15.94:0/2177314691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44001077c0 msgr2=0x7f4400074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43fd400640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44001077c0 0x7f4400074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43fd400640 1 -- 172.21.15.94:0/2177314691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44001a6550 con 0x7f4400102170 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43fde00640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001017b0 0x7f4400073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43fd400640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4400102170 0x7f44000741e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00e9f0 tx=0x7f43ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43f6a00640 1 -- 172.21.15.94:0/2177314691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00ccc0 con 0x7f4400102170 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43f6a00640 1 -- 172.21.15.94:0/2177314691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43ec010910 con 0x7f4400102170 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44001a6840 con 0x7f4400102170 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43f6a00640 1 -- 172.21.15.94:0/2177314691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec018960 con 0x7f4400102170 2024-06-03T01:13:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.788+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44001a6d90 con 0x7f4400102170 2024-06-03T01:13:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 Processor -- start 2024-06-03T01:13:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f43f6a00640 1 -- 172.21.15.94:0/2177314691 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f43ec010ac0 con 0x7f4400102170 2024-06-03T01:13:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 -- start start 2024-06-03T01:13:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f732ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f732d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f733a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5569f7291420 con 0x5569f732e770 2024-06-03T01:13:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569f7238c50 con 0x5569f7337f80 2024-06-03T01:13:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f4407f87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5569f728d3e0 con 0x5569f732d200 2024-06-03T01:13:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f43fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f732d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.792+0000 7f43fd400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f733a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f732d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49302/0 (socket says 172.21.15.94:49302) 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fe800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f732ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 -- 172.21.15.94:0/488601114 learned_addr learned my addr 172.21.15.94:0/488601114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 -- 172.21.15.94:0/488601114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 msgr2=0x5569f733a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 --2- 172.21.15.94:0/488601114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f733a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 -- 172.21.15.94:0/488601114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 msgr2=0x5569f732ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 --2- 172.21.15.94:0/488601114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f732ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 -- 172.21.15.94:0/488601114 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f6fa5210 con 0x5569f732d200 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.793+0000 7f43fde00640 1 --2- 172.21.15.94:0/488601114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f732d680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43f00115f0 tx=0x7f43f0011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d6aa275720cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f43f6a00640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f43e40a2cf0 0x7f43e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f43eb400640 1 -- 172.21.15.94:0/488601114 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f000f8c0 con 0x5569f732d200 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f43eb400640 1 -- 172.21.15.94:0/488601114 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f002a420 con 0x5569f732d200 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f43eb400640 1 -- 172.21.15.94:0/488601114 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0029600 con 0x5569f732d200 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f43f6a00640 1 -- 172.21.15.94:0/2177314691 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f43ec0cfba0 con 0x7f4400102170 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f43fe800640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f43e40a2cf0 0x7f43e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f4407f87740 1 -- 172.21.15.94:0/488601114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 msgr2=0x5569f732d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.794+0000 7f4407f87740 1 --2- 172.21.15.94:0/488601114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f732d680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43f00115f0 tx=0x7f43f0011ab0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f43fe800640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f43e40a2cf0 0x7f43e40a51b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009430 tx=0x7f43f8009360 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 -- 172.21.15.94:0/488601114 shutdown_connections 2024-06-03T01:13:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 --2- 172.21.15.94:0/488601114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f733a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 --2- 172.21.15.94:0/488601114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f732d680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 --2- 172.21.15.94:0/488601114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f732ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 -- 172.21.15.94:0/488601114 >> 172.21.15.94:0/488601114 conn(0x5569f72f5f80 msgr2=0x5569f73297f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 -- 172.21.15.94:0/488601114 shutdown_connections 2024-06-03T01:13:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 -- 172.21.15.94:0/488601114 wait complete. 2024-06-03T01:13:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 -- start start 2024-06-03T01:13:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.795+0000 7f4407f87740 1 Processor -- start 2024-06-03T01:13:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f4407f87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f7334ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f4407f87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7335420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f4407f87740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f7335960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f4407f87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5569f733ad50 con 0x5569f732e770 2024-06-03T01:13:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f4407f87740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5569f724c0b0 con 0x5569f7337f80 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f4407f87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5569f733aed0 con 0x5569f732d200 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f4407f87740 10 client.0 _ll_register_callbacks cb 0x5569f7237f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f43fd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7335420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f43fe800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f7335960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f43fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f7334ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f43fd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7335420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57354/0 (socket says 172.21.15.94:57354) 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f43fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f7334ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49310/0 (socket says 172.21.15.94:49310) 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.796+0000 7f43fd400640 1 -- 172.21.15.94:0/2670496928 learned_addr learned my addr 172.21.15.94:0/2670496928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43fe800640 1 -- 172.21.15.94:0/2670496928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 msgr2=0x5569f7334ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43fe800640 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f7334ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43fe800640 1 -- 172.21.15.94:0/2670496928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 msgr2=0x5569f7335420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43fe800640 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7335420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43fe800640 1 -- 172.21.15.94:0/2670496928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f7071620 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43fd400640 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7335420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43fe800640 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f7335960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43f8031040 tx=0x7f43f802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714f07dc31df9610 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f43dfe00640 1 -- 172.21.15.94:0/2670496928 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8031720 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.797+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f002a420 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f43dfe00640 1 -- 172.21.15.94:0/2670496928 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f43f80318c0 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f43dfe00640 1 -- 172.21.15.94:0/2670496928 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f43f80305c0 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f4407f87740 10 client.25511 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569f7347020 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f43dfe00640 1 -- 172.21.15.94:0/2670496928 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f002a420 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f43dfe00640 1 -- 172.21.15.94:0/2670496928 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f43f8051070 con 0x5569f7337f80 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f4407f87740 10 client.25511 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f4407f87740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f4407f87740 20 client.25511 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.798+0000 7f4407f87740 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73484e0 0x5569f734a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.799+0000 7f4407f87740 4 client.25511 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.799+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f729eb50 con 0x5569f73484e0 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.799+0000 7f43fde00640 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73484e0 0x5569f734a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.799+0000 7f43fde00640 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73484e0 0x5569f734a900 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c96c336c9b080a server_cookie=a28e0aed2ca92370 in_seq=0 out_seq=0 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.799+0000 7f43dfe00640 10 client.25511 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.800+0000 7f43dfe00640 1 -- 172.21.15.94:0/2670496928 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569f729eb50 con 0x5569f73484e0 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.800+0000 7f43dfe00640 10 client.25511 handle_command_reply: tid=0 2024-06-03T01:13:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.800+0000 7f4407f87740 1 client.25511 shutdown 2024-06-03T01:13:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.800+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 msgr2=0x5569f7335960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.800+0000 7f4407f87740 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f7335960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43f8031040 tx=0x7f43f802a8c0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 shutdown_connections 2024-06-03T01:13:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73484e0 0x5569f734a900 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f7337f80 0x5569f7335960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7335420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f43dfe00640 0 client.25511 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 --2- 172.21.15.94:0/2670496928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f732d200 0x5569f7334ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 >> 172.21.15.94:0/2670496928 conn(0x5569f72f5f80 msgr2=0x5569f732f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 shutdown_connections 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 1 -- 172.21.15.94:0/2670496928 wait complete. 2024-06-03T01:13:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.801+0000 7f4407f87740 20 client.25511 trim_cache size 0 max 16384 2024-06-03T01:13:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.819+0000 7f4407f87740 1 Processor -- start 2024-06-03T01:13:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.819+0000 7f4407f87740 1 -- start start 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f4407f87740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f4407f87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7329630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f4407f87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f7339370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f4407f87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44001a8580 con 0x5569f732e770 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f4407f87740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44001aa180 con 0x5569f732d200 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f4407f87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44001a8c50 con 0x5569f7337f80 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f43fd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7329630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f43fde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f43fd400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7329630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57368/0 (socket says 172.21.15.94:57368) 2024-06-03T01:13:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f43fd400640 1 -- 172.21.15.94:0/3034500045 learned_addr learned my addr 172.21.15.94:0/3034500045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f43fde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57776/0 (socket says 172.21.15.94:57776) 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.820+0000 7f43fe800640 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f7339370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43fd400640 1 -- 172.21.15.94:0/3034500045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 msgr2=0x5569f7339370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43fd400640 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f7339370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43fd400640 1 -- 172.21.15.94:0/3034500045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 msgr2=0x5569f73290f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43fd400640 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73290f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43fd400640 1 -- 172.21.15.94:0/3034500045 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f7344020 con 0x5569f732e770 2024-06-03T01:13:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43fd400640 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7329630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f43ec09bab0 tx=0x7f43ec0d3650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d311d00f20a3479 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43eaa00640 1 -- 172.21.15.94:0/3034500045 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec09b710 con 0x5569f732e770 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43eaa00640 1 -- 172.21.15.94:0/3034500045 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43ec0d4040 con 0x5569f732e770 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.821+0000 7f43eaa00640 1 -- 172.21.15.94:0/3034500045 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec09b460 con 0x5569f732e770 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 -- 172.21.15.94:0/3034500045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 msgr2=0x5569f7329630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7329630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f43ec09bab0 tx=0x7f43ec0d3650 comp rx=0 tx=0).stop 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 -- 172.21.15.94:0/3034500045 shutdown_connections 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f7339370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f7329630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 --2- 172.21.15.94:0/3034500045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 -- 172.21.15.94:0/3034500045 >> 172.21.15.94:0/3034500045 conn(0x5569f72f5f80 msgr2=0x5569f7330260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 -- 172.21.15.94:0/3034500045 shutdown_connections 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.822+0000 7f4407f87740 1 -- 172.21.15.94:0/3034500045 wait complete. 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.823+0000 7f4407f87740 1 -- start start 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.823+0000 7f4407f87740 1 Processor -- start 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.823+0000 7f4407f87740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f4407f87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f734f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f4407f87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f734fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f4407f87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44001a99f0 con 0x5569f732e770 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f4407f87740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44001a9320 con 0x5569f732d200 2024-06-03T01:13:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f4407f87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4400073140 con 0x5569f7337f80 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f4407f87740 10 client.0 _ll_register_callbacks cb 0x7f43f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57778/0 (socket says 172.21.15.94:57778) 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 -- 172.21.15.94:0/4109919871 learned_addr learned my addr 172.21.15.94:0/4109919871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fe800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f734fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fd400640 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f734f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 -- 172.21.15.94:0/4109919871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 msgr2=0x5569f734fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f734fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 -- 172.21.15.94:0/4109919871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 msgr2=0x5569f734f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f734f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 -- 172.21.15.94:0/4109919871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f73503e0 con 0x5569f732d200 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fd400640 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f734f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:13:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43fde00640 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73406b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f43f002ad70 tx=0x7f43f0005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620b32781211ab80 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.824+0000 7f43dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.825+0000 7f43dfe00640 1 -- 172.21.15.94:0/4109919871 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0005240 con 0x5569f732d200 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.825+0000 7f43dfe00640 1 -- 172.21.15.94:0/4109919871 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f43f00053e0 con 0x5569f732d200 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.825+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43ec0d4040 con 0x5569f732d200 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.825+0000 7f43dfe00640 1 -- 172.21.15.94:0/4109919871 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f43f004c050 con 0x5569f732d200 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.825+0000 7f4407f87740 10 client.25517 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.825+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569f733ef70 con 0x5569f732d200 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.825+0000 7f43dfe00640 1 -- 172.21.15.94:0/4109919871 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43ec0d4040 con 0x5569f732d200 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f43dfe00640 1 -- 172.21.15.94:0/4109919871 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f43f000f990 con 0x5569f732d200 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f4407f87740 10 client.25517 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f4407f87740 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f4407f87740 20 client.25517 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f4407f87740 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73b34c0 0x5569f73b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f4407f87740 4 client.25517 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:13:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569f7298ea0 con 0x5569f73b34c0 2024-06-03T01:13:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f43fd400640 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73b34c0 0x5569f73b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f43dfe00640 10 client.25517 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.826+0000 7f43fd400640 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73b34c0 0x5569f73b58e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8428dc0a1c6b0bff server_cookie=26dbfd13ce18cd4b in_seq=0 out_seq=0 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f43dfe00640 1 -- 172.21.15.94:0/4109919871 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5569f7298ea0 con 0x5569f73b34c0 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f43dfe00640 10 client.25517 handle_command_reply: tid=0 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 client.25517 shutdown 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 msgr2=0x5569f73406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73406b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f43f002ad70 tx=0x7f43f0005d10 comp rx=0 tx=0).stop 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 shutdown_connections 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5569f7337f80 0x5569f734fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569f732e770 0x5569f734f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5569f73b34c0 0x5569f73b58e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 --2- 172.21.15.94:0/4109919871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5569f732d200 0x5569f73406b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f43dfe00640 0 client.25517 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 >> 172.21.15.94:0/4109919871 conn(0x5569f72f5f80 msgr2=0x5569f7343010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 shutdown_connections 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.863+0000 7f4407f87740 1 -- 172.21.15.94:0/4109919871 wait complete. 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f4407f87740 20 client.25517 trim_cache size 0 max 16384 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:13:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:13:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f43e40a2cf0 msgr2=0x7f43e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f43e40a2cf0 0x7f43e40a51b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009430 tx=0x7f43f8009360 comp rx=0 tx=0).stop 2024-06-03T01:13:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4400102170 msgr2=0x7f44000741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4400102170 0x7f44000741e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00e9f0 tx=0x7f43ec00eeb0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 shutdown_connections 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f43e40a2cf0 0x7f43e40a51b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44001077c0 0x7f4400074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4400102170 0x7f44000741e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 --2- 172.21.15.94:0/2177314691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44001017b0 0x7f4400073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.864+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 >> 172.21.15.94:0/2177314691 conn(0x7f44000fd2e0 msgr2=0x7f44000fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.865+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 shutdown_connections 2024-06-03T01:13:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:02.865+0000 7f43ff200640 1 -- 172.21.15.94:0/2177314691 wait complete. 2024-06-03T01:13:03.061 INFO:tasks.workunit.client.0.smithi094.stdout: 1 41036 3.38 MB/sec warmup 70 sec latency 101.839 ms 2024-06-03T01:13:04.061 INFO:tasks.workunit.client.0.smithi094.stdout: 1 42545 3.50 MB/sec warmup 71 sec latency 16.039 ms 2024-06-03T01:13:04.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:03 smithi094 ceph-mon[32119]: pgmap v84: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 655 KiB/s wr, 107 op/s 2024-06-03T01:13:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:03 smithi179 ceph-mon[37041]: pgmap v84: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 655 KiB/s wr, 107 op/s 2024-06-03T01:13:04.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:03 smithi107 ceph-mon[36622]: pgmap v84: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 655 KiB/s wr, 107 op/s 2024-06-03T01:13:05.061 INFO:tasks.workunit.client.0.smithi094.stdout: 1 43200 3.48 MB/sec warmup 72 sec latency 16.290 ms 2024-06-03T01:13:06.061 INFO:tasks.workunit.client.0.smithi094.stdout: 1 43398 3.44 MB/sec warmup 73 sec latency 771.304 ms 2024-06-03T01:13:06.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:05 smithi179 ceph-mon[37041]: pgmap v85: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 616 KiB/s wr, 91 op/s 2024-06-03T01:13:06.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:05 smithi107 ceph-mon[36622]: pgmap v85: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 616 KiB/s wr, 91 op/s 2024-06-03T01:13:06.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:05 smithi094 ceph-mon[32119]: pgmap v85: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 616 KiB/s wr, 91 op/s 2024-06-03T01:13:07.062 INFO:tasks.workunit.client.0.smithi094.stdout: 1 45028 3.48 MB/sec warmup 74 sec latency 15.491 ms 2024-06-03T01:13:08.062 INFO:tasks.workunit.client.0.smithi094.stdout: 1 46373 3.59 MB/sec warmup 75 sec latency 14.069 ms 2024-06-03T01:13:08.152 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:07 smithi094 ceph-mon[32119]: pgmap v86: 129 pgs: 129 active+clean; 28 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 777 KiB/s wr, 141 op/s 2024-06-03T01:13:08.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:07 smithi107 ceph-mon[36622]: pgmap v86: 129 pgs: 129 active+clean; 28 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 777 KiB/s wr, 141 op/s 2024-06-03T01:13:08.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:07 smithi179 ceph-mon[37041]: pgmap v86: 129 pgs: 129 active+clean; 28 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 777 KiB/s wr, 141 op/s 2024-06-03T01:13:09.062 INFO:tasks.workunit.client.0.smithi094.stdout: 1 46814 3.55 MB/sec warmup 76 sec latency 219.748 ms 2024-06-03T01:13:10.062 INFO:tasks.workunit.client.0.smithi094.stdout: 1 46814 3.50 MB/sec warmup 77 sec latency 1219.890 ms 2024-06-03T01:13:10.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:09 smithi107 ceph-mon[36622]: pgmap v87: 129 pgs: 129 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 739 KiB/s wr, 150 op/s 2024-06-03T01:13:10.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:09 smithi094 ceph-mon[32119]: pgmap v87: 129 pgs: 129 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 739 KiB/s wr, 150 op/s 2024-06-03T01:13:10.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:09 smithi179 ceph-mon[37041]: pgmap v87: 129 pgs: 129 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 739 KiB/s wr, 150 op/s 2024-06-03T01:13:11.062 INFO:tasks.workunit.client.0.smithi094.stdout: 1 46829 3.46 MB/sec warmup 78 sec latency 2201.840 ms 2024-06-03T01:13:12.062 INFO:tasks.workunit.client.0.smithi094.stdout: 1 48474 3.50 MB/sec warmup 79 sec latency 14.714 ms 2024-06-03T01:13:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:11 smithi107 ceph-mon[36622]: pgmap v88: 129 pgs: 129 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 KiB/s wr, 134 op/s 2024-06-03T01:13:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:11 smithi094 ceph-mon[32119]: pgmap v88: 129 pgs: 129 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 KiB/s wr, 134 op/s 2024-06-03T01:13:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:11 smithi179 ceph-mon[37041]: pgmap v88: 129 pgs: 129 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 KiB/s wr, 134 op/s 2024-06-03T01:13:13.062 INFO:tasks.workunit.client.0.smithi094.stdout: 1 49821 3.59 MB/sec warmup 80 sec latency 15.021 ms 2024-06-03T01:13:14.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50355 3.57 MB/sec warmup 81 sec latency 126.521 ms 2024-06-03T01:13:14.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:13 smithi107 ceph-mon[36622]: pgmap v89: 129 pgs: 129 active+clean; 32 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 779 KiB/s wr, 178 op/s 2024-06-03T01:13:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:13 smithi094 ceph-mon[32119]: pgmap v89: 129 pgs: 129 active+clean; 32 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 779 KiB/s wr, 178 op/s 2024-06-03T01:13:14.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:13 smithi179 ceph-mon[37041]: pgmap v89: 129 pgs: 129 active+clean; 32 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 779 KiB/s wr, 178 op/s 2024-06-03T01:13:15.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50355 3.52 MB/sec warmup 82 sec latency 1126.669 ms 2024-06-03T01:13:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 -- 172.21.15.94:0/923491079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 msgr2=0x7f20781041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 --2- 172.21.15.94:0/923491079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f20781041f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2068009900 tx=0x7f206802a410 comp rx=0 tx=0).stop 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 -- 172.21.15.94:0/923491079 shutdown_connections 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 --2- 172.21.15.94:0/923491079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2078108ee0 0x7f2078110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 --2- 172.21.15.94:0/923491079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781047c0 0x7f2078108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 --2- 172.21.15.94:0/923491079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f20781041f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 -- 172.21.15.94:0/923491079 >> 172.21.15.94:0/923491079 conn(0x7f20780ff670 msgr2=0x7f2078101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 -- 172.21.15.94:0/923491079 shutdown_connections 2024-06-03T01:13:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.007+0000 7f207d200640 1 -- 172.21.15.94:0/923491079 wait complete. 2024-06-03T01:13:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.008+0000 7f207d200640 1 Processor -- start 2024-06-03T01:13:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.008+0000 7f207d200640 1 -- start start 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f207d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f207819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f207d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2078108ee0 0x7f207819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f207d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f207819ec40 0x7f207819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f207d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2078113f90 con 0x7f2078103e10 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f207d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2078113e10 con 0x7f2078108ee0 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f207d200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2078114110 con 0x7f207819ec40 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f2075e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f207819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f2075e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f207819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39442/0 (socket says 172.21.15.94:39442) 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f2075e00640 1 -- 172.21.15.94:0/1007523519 learned_addr learned my addr 172.21.15.94:0/1007523519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f2075400640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2078108ee0 0x7f207819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.009+0000 7f2076800640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f207819ec40 0x7f207819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f2075e00640 1 -- 172.21.15.94:0/1007523519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f207819ec40 msgr2=0x7f207819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f2075e00640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f207819ec40 0x7f207819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f2075e00640 1 -- 172.21.15.94:0/1007523519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2078108ee0 msgr2=0x7f207819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f2075e00640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2078108ee0 0x7f207819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f2075e00640 1 -- 172.21.15.94:0/1007523519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207819f730 con 0x7f2078103e10 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f2075400640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2078108ee0 0x7f207819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f2075e00640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f207819a3d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2068006010 tx=0x7f2068004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f206ea00640 1 -- 172.21.15.94:0/1007523519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206803f030 con 0x7f2078103e10 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f206ea00640 1 -- 172.21.15.94:0/1007523519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2068004440 con 0x7f2078103e10 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207819f9c0 con 0x7f2078103e10 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.010+0000 7f206ea00640 1 -- 172.21.15.94:0/1007523519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068035070 con 0x7f2078103e10 2024-06-03T01:13:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.011+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781a7180 con 0x7f2078103e10 2024-06-03T01:13:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.014+0000 7f20801c3740 1 Processor -- start 2024-06-03T01:13:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.014+0000 7f206ea00640 1 -- 172.21.15.94:0/1007523519 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2068042030 con 0x7f2078103e10 2024-06-03T01:13:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.014+0000 7f20801c3740 1 -- start start 2024-06-03T01:13:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f20801c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451e5b0 0x55ea1451cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f20801c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1451d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f20801c3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1452a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f20801c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ea14481420 con 0x55ea1451d040 2024-06-03T01:13:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f20801c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea14428c50 con 0x55ea1451e5b0 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f20801c3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ea1447d3e0 con 0x55ea14527dc0 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2075e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1451d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2076800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451e5b0 0x55ea1451cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2075e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1451d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39458/0 (socket says 172.21.15.94:39458) 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2075e00640 1 -- 172.21.15.94:0/3713560471 learned_addr learned my addr 172.21.15.94:0/3713560471 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2075400640 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1452a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2076800640 1 -- 172.21.15.94:0/3713560471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 msgr2=0x55ea1452a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2076800640 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1452a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2076800640 1 -- 172.21.15.94:0/3713560471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 msgr2=0x55ea1451d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2076800640 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1451d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2076800640 1 -- 172.21.15.94:0/3713560471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea14195210 con 0x55ea1451e5b0 2024-06-03T01:13:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2075400640 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1452a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.015+0000 7f2075e00640 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1451d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.016+0000 7f2076800640 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451e5b0 0x55ea1451cb00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2070009f30 tx=0x7f20700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc31ea11315fcf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.016+0000 7f206ea00640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f205c0a2da0 0x7f205c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.016+0000 7f2063400640 1 -- 172.21.15.94:0/3713560471 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070004d20 con 0x55ea1451e5b0 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.016+0000 7f2063400640 1 -- 172.21.15.94:0/3713560471 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2070004050 con 0x55ea1451e5b0 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.016+0000 7f2063400640 1 -- 172.21.15.94:0/3713560471 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070005220 con 0x55ea1451e5b0 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.016+0000 7f206ea00640 1 -- 172.21.15.94:0/1007523519 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f20680efb50 con 0x7f2078103e10 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.016+0000 7f2075e00640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f205c0a2da0 0x7f205c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 -- 172.21.15.94:0/3713560471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451e5b0 msgr2=0x55ea1451cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451e5b0 0x55ea1451cb00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2070009f30 tx=0x7f20700118d0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 -- 172.21.15.94:0/3713560471 shutdown_connections 2024-06-03T01:13:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1452a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1451d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 --2- 172.21.15.94:0/3713560471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451e5b0 0x55ea1451cb00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 -- 172.21.15.94:0/3713560471 >> 172.21.15.94:0/3713560471 conn(0x55ea144e5f80 msgr2=0x55ea14519630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f2075e00640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f205c0a2da0 0x7f205c0a5260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2068032c00 tx=0x7f20680f3600 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 -- 172.21.15.94:0/3713560471 shutdown_connections 2024-06-03T01:13:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.017+0000 7f20801c3740 1 -- 172.21.15.94:0/3713560471 wait complete. 2024-06-03T01:13:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 -- start start 2024-06-03T01:13:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 Processor -- start 2024-06-03T01:13:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451d040 0x55ea14524e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea14525370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ea1452ab90 con 0x55ea1451e5b0 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea1443c0b0 con 0x55ea14527dc0 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.018+0000 7f20801c3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ea1452ad10 con 0x55ea1451d040 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f20801c3740 10 client.0 _ll_register_callbacks cb 0x55ea14427f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2076800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451d040 0x55ea14524e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea14525370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2076800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451d040 0x55ea14524e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51152/0 (socket says 172.21.15.94:51152) 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea14525370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39470/0 (socket says 172.21.15.94:39470) 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 -- 172.21.15.94:0/2342924410 learned_addr learned my addr 172.21.15.94:0/2342924410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2076800640 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 -- 172.21.15.94:0/2342924410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451d040 msgr2=0x55ea14524e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451d040 0x55ea14524e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 -- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 msgr2=0x55ea145258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2075400640 1 -- 172.21.15.94:0/2342924410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea14261620 con 0x55ea1451e5b0 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.019+0000 7f2076800640 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f2075400640 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea14525370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2064012960 tx=0x7f2064012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb8555c810fad9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f2057e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f2057e00640 1 -- 172.21.15.94:0/2342924410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206400fa00 con 0x55ea1451e5b0 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f2057e00640 1 -- 172.21.15.94:0/2342924410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f206400fba0 con 0x55ea1451e5b0 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2070004050 con 0x55ea1451e5b0 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f2057e00640 1 -- 172.21.15.94:0/2342924410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2064025050 con 0x55ea1451e5b0 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f20801c3740 10 client.15405 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea14536fb0 con 0x55ea1451e5b0 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.020+0000 7f2057e00640 1 -- 172.21.15.94:0/2342924410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2070004050 con 0x55ea1451e5b0 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f2057e00640 1 -- 172.21.15.94:0/2342924410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f2064010430 con 0x55ea1451e5b0 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f20801c3740 10 client.15405 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f20801c3740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f20801c3740 20 client.15405 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f20801c3740 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea14538530 0x55ea1453a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f20801c3740 4 client.15405 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1448eb50 con 0x55ea14538530 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.021+0000 7f2076800640 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea14538530 0x55ea1453a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.022+0000 7f2076800640 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea14538530 0x55ea1453a920 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12726fc0e444655d server_cookie=35878d2bd66d0ced in_seq=0 out_seq=0 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.022+0000 7f2057e00640 10 client.15405 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f2057e00640 1 -- 172.21.15.94:0/2342924410 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea1448eb50 con 0x55ea14538530 2024-06-03T01:13:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f2057e00640 10 client.15405 handle_command_reply: tid=0 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 client.15405 shutdown 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 msgr2=0x55ea14525370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea14525370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2064012960 tx=0x7f2064012e20 comp rx=0 tx=0).stop 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 shutdown_connections 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea14538530 0x55ea1453a920 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea14525370 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f2057e00640 0 client.15405 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 --2- 172.21.15.94:0/2342924410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451d040 0x55ea14524e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 >> 172.21.15.94:0/2342924410 conn(0x55ea144e5f80 msgr2=0x55ea1451f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 shutdown_connections 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 1 -- 172.21.15.94:0/2342924410 wait complete. 2024-06-03T01:13:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.023+0000 7f20801c3740 20 client.15405 trim_cache size 0 max 16384 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 Processor -- start 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 -- start start 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451d040 0x55ea14523f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea145256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1451f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b0e20 con 0x55ea1451e5b0 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b2a20 con 0x55ea1451d040 2024-06-03T01:13:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.041+0000 7f20801c3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b14f0 con 0x55ea14527dc0 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451d040 0x55ea14523f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451d040 0x55ea14523f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:55166/0 (socket says 172.21.15.94:55166) 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 -- 172.21.15.94:0/1303464550 learned_addr learned my addr 172.21.15.94:0/1303464550 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2075400640 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea145256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1451f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 -- 172.21.15.94:0/1303464550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 msgr2=0x55ea1451f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1451f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 -- 172.21.15.94:0/1303464550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 msgr2=0x55ea145256d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea145256d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 -- 172.21.15.94:0/1303464550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea14514ff0 con 0x55ea1451d040 2024-06-03T01:13:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2075400640 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea145256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.042+0000 7f2076800640 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451d040 0x55ea14523f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2070005fd0 tx=0x7f2070009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1814f4e3af97f95 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.043+0000 7f2062a00640 1 -- 172.21.15.94:0/1303464550 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070004490 con 0x55ea1451d040 2024-06-03T01:13:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.043+0000 7f2062a00640 1 -- 172.21.15.94:0/1303464550 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2070004630 con 0x55ea1451d040 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.043+0000 7f2062a00640 1 -- 172.21.15.94:0/1303464550 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207000f480 con 0x55ea1451d040 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.043+0000 7f20801c3740 1 -- 172.21.15.94:0/1303464550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451d040 msgr2=0x55ea14523f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.043+0000 7f20801c3740 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451d040 0x55ea14523f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2070005fd0 tx=0x7f2070009d00 comp rx=0 tx=0).stop 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.043+0000 7f20801c3740 1 -- 172.21.15.94:0/1303464550 shutdown_connections 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea14527dc0 0x55ea1451f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451e5b0 0x55ea145256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 --2- 172.21.15.94:0/1303464550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea1451d040 0x55ea14523f90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 -- 172.21.15.94:0/1303464550 >> 172.21.15.94:0/1303464550 conn(0x55ea144e5f80 msgr2=0x55ea1451eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 -- 172.21.15.94:0/1303464550 shutdown_connections 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 -- 172.21.15.94:0/1303464550 wait complete. 2024-06-03T01:13:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 -- start start 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.044+0000 7f20801c3740 1 Processor -- start 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f20801c3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1452b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f20801c3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451e5b0 0x55ea1452baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f20801c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f20801c3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b2290 con 0x55ea1451d040 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f20801c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20781b1bc0 con 0x55ea14527dc0 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f20801c3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20781aee00 con 0x55ea1451e5b0 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f20801c3740 10 client.0 _ll_register_callbacks cb 0x7f2064009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2076800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1452b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451e5b0 0x55ea1452baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451e5b0 0x55ea1452baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51158/0 (socket says 172.21.15.94:51158) 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 -- 172.21.15.94:0/4162841573 learned_addr learned my addr 172.21.15.94:0/4162841573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2076800640 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 -- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 msgr2=0x55ea145a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 -- 172.21.15.94:0/4162841573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 msgr2=0x55ea1452b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1452b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2075400640 1 -- 172.21.15.94:0/4162841573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea145a16f0 con 0x55ea1451e5b0 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.045+0000 7f2076800640 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1452b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f2075400640 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451e5b0 0x55ea1452baa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20640058c0 tx=0x7f20640125b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bf4b354cc88656 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f2057e00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:13:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f2057e00640 1 -- 172.21.15.94:0/4162841573 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064022040 con 0x55ea1451e5b0 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2070004630 con 0x55ea1451e5b0 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f2057e00640 1 -- 172.21.15.94:0/4162841573 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f2064010430 con 0x55ea1451e5b0 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f2057e00640 1 -- 172.21.15.94:0/4162841573 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2064010630 con 0x55ea1451e5b0 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f20801c3740 10 client.25549 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ea145a28e0 con 0x55ea1451e5b0 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.046+0000 7f2057e00640 1 -- 172.21.15.94:0/4162841573 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2070004630 con 0x55ea1451e5b0 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f2057e00640 1 -- 172.21.15.94:0/4162841573 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f20640035f0 con 0x55ea1451e5b0 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f20801c3740 10 client.25549 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f20801c3740 10 client.25549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f20801c3740 20 client.25549 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f20801c3740 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea145a30b0 0x55ea145a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f20801c3740 4 client.25549 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:13:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea14488ea0 con 0x55ea145a30b0 2024-06-03T01:13:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.047+0000 7f2076800640 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea145a30b0 0x55ea145a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.048+0000 7f2076800640 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea145a30b0 0x55ea145a54d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b66795b83b917920 server_cookie=2d13c19d23c9ffeb in_seq=0 out_seq=0 2024-06-03T01:13:16.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.048+0000 7f2057e00640 10 client.25549 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:16.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50365 3.48 MB/sec warmup 83 sec latency 2107.532 ms 2024-06-03T01:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.091+0000 7f2057e00640 1 -- 172.21.15.94:0/4162841573 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea14488ea0 con 0x55ea145a30b0 2024-06-03T01:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.091+0000 7f2057e00640 10 client.25549 handle_command_reply: tid=0 2024-06-03T01:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.091+0000 7f20801c3740 1 client.25549 shutdown 2024-06-03T01:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.091+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451e5b0 msgr2=0x55ea1452baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.091+0000 7f20801c3740 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451e5b0 0x55ea1452baa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20640058c0 tx=0x7f20640125b0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 shutdown_connections 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ea145a30b0 0x55ea145a54d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea14527dc0 0x55ea145a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ea1451e5b0 0x55ea1452baa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 --2- 172.21.15.94:0/4162841573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea1451d040 0x55ea1452b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f2057e00640 0 client.25549 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 >> 172.21.15.94:0/4162841573 conn(0x55ea144e5f80 msgr2=0x55ea14518810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 shutdown_connections 2024-06-03T01:13:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 1 -- 172.21.15.94:0/4162841573 wait complete. 2024-06-03T01:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.092+0000 7f20801c3740 20 client.25549 trim_cache size 0 max 16384 2024-06-03T01:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:13:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f205c0a2da0 msgr2=0x7f205c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f205c0a2da0 0x7f205c0a5260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2068032c00 tx=0x7f20680f3600 comp rx=0 tx=0).stop 2024-06-03T01:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 msgr2=0x7f207819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f207819a3d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2068006010 tx=0x7f2068004040 comp rx=0 tx=0).stop 2024-06-03T01:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 shutdown_connections 2024-06-03T01:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f205c0a2da0 0x7f205c0a5260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f207819ec40 0x7f207819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2078108ee0 0x7f207819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.093+0000 7f207d200640 1 --2- 172.21.15.94:0/1007523519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2078103e10 0x7f207819a3d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.094+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 >> 172.21.15.94:0/1007523519 conn(0x7f20780ff670 msgr2=0x7f20781001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.094+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 shutdown_connections 2024-06-03T01:13:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:16.094+0000 7f207d200640 1 -- 172.21.15.94:0/1007523519 wait complete. 2024-06-03T01:13:16.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:15 smithi107 ceph-mon[36622]: pgmap v90: 129 pgs: 129 active+clean; 32 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 553 KiB/s wr, 119 op/s 2024-06-03T01:13:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:15 smithi094 ceph-mon[32119]: pgmap v90: 129 pgs: 129 active+clean; 32 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 553 KiB/s wr, 119 op/s 2024-06-03T01:13:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:15 smithi179 ceph-mon[37041]: pgmap v90: 129 pgs: 129 active+clean; 32 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 553 KiB/s wr, 119 op/s 2024-06-03T01:13:17.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50411 3.44 MB/sec warmup 84 sec latency 927.900 ms 2024-06-03T01:13:18.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50411 3.40 MB/sec warmup 85 sec latency 1927.989 ms 2024-06-03T01:13:18.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:17 smithi107 ceph-mon[36622]: pgmap v91: 129 pgs: 129 active+clean; 30 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 730 KiB/s wr, 170 op/s 2024-06-03T01:13:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:17 smithi094 ceph-mon[32119]: pgmap v91: 129 pgs: 129 active+clean; 30 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 730 KiB/s wr, 170 op/s 2024-06-03T01:13:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:17 smithi179 ceph-mon[37041]: pgmap v91: 129 pgs: 129 active+clean; 30 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 730 KiB/s wr, 170 op/s 2024-06-03T01:13:19.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50411 3.36 MB/sec warmup 86 sec latency 2928.137 ms 2024-06-03T01:13:20.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50411 3.32 MB/sec warmup 87 sec latency 3928.284 ms 2024-06-03T01:13:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:19 smithi107 ceph-mon[36622]: pgmap v92: 129 pgs: 129 active+clean; 30 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 570 KiB/s wr, 123 op/s 2024-06-03T01:13:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:19 smithi094 ceph-mon[32119]: pgmap v92: 129 pgs: 129 active+clean; 30 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 570 KiB/s wr, 123 op/s 2024-06-03T01:13:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:19 smithi179 ceph-mon[37041]: pgmap v92: 129 pgs: 129 active+clean; 30 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 570 KiB/s wr, 123 op/s 2024-06-03T01:13:21.063 INFO:tasks.workunit.client.0.smithi094.stdout: 1 50423 3.28 MB/sec warmup 88 sec latency 4909.425 ms 2024-06-03T01:13:22.064 INFO:tasks.workunit.client.0.smithi094.stdout: 1 52086 3.32 MB/sec warmup 89 sec latency 15.907 ms 2024-06-03T01:13:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:21 smithi107 ceph-mon[36622]: pgmap v93: 129 pgs: 129 active+clean; 30 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 408 KiB/s wr, 97 op/s 2024-06-03T01:13:22.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:21 smithi094 ceph-mon[32119]: pgmap v93: 129 pgs: 129 active+clean; 30 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 408 KiB/s wr, 97 op/s 2024-06-03T01:13:22.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:21 smithi179 ceph-mon[37041]: pgmap v93: 129 pgs: 129 active+clean; 30 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 408 KiB/s wr, 97 op/s 2024-06-03T01:13:23.064 INFO:tasks.workunit.client.0.smithi094.stdout: 1 53463 3.42 MB/sec warmup 90 sec latency 14.268 ms 2024-06-03T01:13:24.064 INFO:tasks.workunit.client.0.smithi094.stdout: 1 54027 3.39 MB/sec warmup 91 sec latency 58.490 ms 2024-06-03T01:13:24.156 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:23 smithi179 ceph-mon[37041]: pgmap v94: 129 pgs: 129 active+clean; 30 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 487 KiB/s wr, 114 op/s 2024-06-03T01:13:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:23 smithi107 ceph-mon[36622]: pgmap v94: 129 pgs: 129 active+clean; 30 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 487 KiB/s wr, 114 op/s 2024-06-03T01:13:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:23 smithi094 ceph-mon[32119]: pgmap v94: 129 pgs: 129 active+clean; 30 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 487 KiB/s wr, 114 op/s 2024-06-03T01:13:25.064 INFO:tasks.workunit.client.0.smithi094.stdout: 1 54027 3.35 MB/sec warmup 92 sec latency 1058.633 ms 2024-06-03T01:13:26.064 INFO:tasks.workunit.client.0.smithi094.stdout: 1 54039 3.31 MB/sec warmup 93 sec latency 2041.066 ms 2024-06-03T01:13:26.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:25 smithi107 ceph-mon[36622]: pgmap v95: 129 pgs: 129 active+clean; 30 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 257 KiB/s wr, 70 op/s 2024-06-03T01:13:26.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:25 smithi094 ceph-mon[32119]: pgmap v95: 129 pgs: 129 active+clean; 30 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 257 KiB/s wr, 70 op/s 2024-06-03T01:13:26.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:25 smithi179 ceph-mon[37041]: pgmap v95: 129 pgs: 129 active+clean; 30 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 257 KiB/s wr, 70 op/s 2024-06-03T01:13:27.064 INFO:tasks.workunit.client.0.smithi094.stdout: 1 55666 3.35 MB/sec warmup 94 sec latency 34.986 ms 2024-06-03T01:13:28.064 INFO:tasks.workunit.client.0.smithi094.stdout: 1 57025 3.44 MB/sec warmup 95 sec latency 17.471 ms 2024-06-03T01:13:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:27 smithi107 ceph-mon[36622]: pgmap v96: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 486 KiB/s wr, 112 op/s 2024-06-03T01:13:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:27 smithi094 ceph-mon[32119]: pgmap v96: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 486 KiB/s wr, 112 op/s 2024-06-03T01:13:28.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:27 smithi179 ceph-mon[37041]: pgmap v96: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 486 KiB/s wr, 112 op/s 2024-06-03T01:13:29.065 INFO:tasks.workunit.client.0.smithi094.stdout: 1 57571 3.41 MB/sec warmup 96 sec latency 135.137 ms 2024-06-03T01:13:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 -- 172.21.15.94:0/2853860414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c100f40 msgr2=0x7f8c2c10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 --2- 172.21.15.94:0/2853860414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c100f40 0x7f8c2c10dc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009860 tx=0x7f8c1802a340 comp rx=0 tx=0).stop 2024-06-03T01:13:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 -- 172.21.15.94:0/2853860414 shutdown_connections 2024-06-03T01:13:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 --2- 172.21.15.94:0/2853860414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c10e140 0x7f8c2c110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 --2- 172.21.15.94:0/2853860414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c100f40 0x7f8c2c10dc00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 --2- 172.21.15.94:0/2853860414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c2c100590 0x7f8c2c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 -- 172.21.15.94:0/2853860414 >> 172.21.15.94:0/2853860414 conn(0x7f8c2c0fc1f0 msgr2=0x7f8c2c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 -- 172.21.15.94:0/2853860414 shutdown_connections 2024-06-03T01:13:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.233+0000 7f8c32e00640 1 -- 172.21.15.94:0/2853860414 wait complete. 2024-06-03T01:13:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.234+0000 7f8c32e00640 1 Processor -- start 2024-06-03T01:13:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.234+0000 7f8c32e00640 1 -- start start 2024-06-03T01:13:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c32e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c100590 0x7f8c2c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c32e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c2c100f40 0x7f8c2c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c32e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c10e140 0x7f8c2c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c32e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c113df0 con 0x7f8c2c100f40 2024-06-03T01:13:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c32e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c113c70 con 0x7f8c2c100590 2024-06-03T01:13:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c32e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c113f70 con 0x7f8c2c10e140 2024-06-03T01:13:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c100590 0x7f8c2c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c100590 0x7f8c2c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45778/0 (socket says 172.21.15.94:45778) 2024-06-03T01:13:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c31000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c2c100f40 0x7f8c2c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c31a00640 1 -- 172.21.15.94:0/835056989 learned_addr learned my addr 172.21.15.94:0/835056989 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.235+0000 7f8c32400640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c10e140 0x7f8c2c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c31a00640 1 -- 172.21.15.94:0/835056989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c10e140 msgr2=0x7f8c2c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c31a00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c10e140 0x7f8c2c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c31a00640 1 -- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c2c100f40 msgr2=0x7f8c2c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c31a00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c2c100f40 0x7f8c2c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c31a00640 1 -- 172.21.15.94:0/835056989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c19f350 con 0x7f8c2c100590 2024-06-03T01:13:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c31a00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c100590 0x7f8c2c19a3b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00efb0 tx=0x7f8c1c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c2a000640 1 -- 172.21.15.94:0/835056989 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c01b070 con 0x7f8c2c100590 2024-06-03T01:13:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2c19f640 con 0x7f8c2c100590 2024-06-03T01:13:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c2a000640 1 -- 172.21.15.94:0/835056989 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c00ba50 con 0x7f8c2c100590 2024-06-03T01:13:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c2a000640 1 -- 172.21.15.94:0/835056989 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c016c00 con 0x7f8c2c100590 2024-06-03T01:13:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.236+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2c1a6ed0 con 0x7f8c2c100590 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 Processor -- start 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 -- start start 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c2a000640 1 -- 172.21.15.94:0/835056989 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8c1c01e070 con 0x7f8c2c100590 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de11680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd75420 con 0x563d6de1bf80 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd1cc50 con 0x563d6de12770 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.241+0000 7f8c37252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd713e0 con 0x563d6de11200 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de11680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c32400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55720/0 (socket says 172.21.15.94:55720) 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de11680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:47622/0 (socket says 172.21.15.94:47622) 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 -- 172.21.15.94:0/2431094521 learned_addr learned my addr 172.21.15.94:0/2431094521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c32400640 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de10cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45790/0 (socket says 172.21.15.94:45790) 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 -- 172.21.15.94:0/2431094521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 msgr2=0x563d6de11680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de11680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 -- 172.21.15.94:0/2431094521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 msgr2=0x563d6de10cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de10cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 -- 172.21.15.94:0/2431094521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6da89210 con 0x563d6de1bf80 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31a00640 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de11680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c32400640 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c31000640 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1e370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009780 tx=0x7f8c18009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7801df2d25c08eff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.242+0000 7f8c2a000640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f8c100a2cf0 0x7f8c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.243+0000 7f8c2a000640 1 -- 172.21.15.94:0/835056989 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f8c1c0cf6f0 con 0x7f8c2c100590 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.243+0000 7f8c22a00640 1 -- 172.21.15.94:0/2431094521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18041020 con 0x563d6de1bf80 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.243+0000 7f8c32400640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f8c100a2cf0 0x7f8c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.243+0000 7f8c22a00640 1 -- 172.21.15.94:0/2431094521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c18030e50 con 0x563d6de1bf80 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.243+0000 7f8c22a00640 1 -- 172.21.15.94:0/2431094521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18037040 con 0x563d6de1bf80 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 -- 172.21.15.94:0/2431094521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 msgr2=0x563d6de1e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1e370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009780 tx=0x7f8c18009510 comp rx=0 tx=0).stop 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c32400640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f8c100a2cf0 0x7f8c100a51b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c102770 tx=0x7f8c24005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 -- 172.21.15.94:0/2431094521 shutdown_connections 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1e370 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de11680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 --2- 172.21.15.94:0/2431094521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 -- 172.21.15.94:0/2431094521 >> 172.21.15.94:0/2431094521 conn(0x563d6ddd9f80 msgr2=0x563d6de0d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 -- 172.21.15.94:0/2431094521 shutdown_connections 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 -- 172.21.15.94:0/2431094521 wait complete. 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.244+0000 7f8c37252740 1 -- start start 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 1 Processor -- start 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de11200 0x563d6de18f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de12770 0x563d6de19440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de19980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563d6de1ecf0 con 0x563d6de11200 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563d6dd300b0 con 0x563d6de1bf80 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d6de1ee70 con 0x563d6de12770 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c37252740 10 client.0 _ll_register_callbacks cb 0x563d6dd1bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.245+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de11200 0x563d6de18f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de11200 0x563d6de18f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55736/0 (socket says 172.21.15.94:55736) 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 -- 172.21.15.94:0/3253709231 learned_addr learned my addr 172.21.15.94:0/3253709231 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c32400640 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de19980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31000640 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de12770 0x563d6de19440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 -- 172.21.15.94:0/3253709231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de12770 msgr2=0x563d6de19440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de12770 0x563d6de19440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 -- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 msgr2=0x563d6de19980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de19980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 -- 172.21.15.94:0/3253709231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6db55620 con 0x563d6de11200 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c31a00640 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de11200 0x563d6de18f00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c09b660 tx=0x7f8c1c097cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5046c87a6d0dfbda server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c17400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c17400640 1 -- 172.21.15.94:0/3253709231 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c09b2c0 con 0x563d6de11200 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c17400640 1 -- 172.21.15.94:0/3253709231 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f8c1c00bc00 con 0x563d6de11200 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.246+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c18030e50 con 0x563d6de11200 2024-06-03T01:13:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c17400640 1 -- 172.21.15.94:0/3253709231 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c1c09b960 con 0x563d6de11200 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 10 client.15423 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d6de2b010 con 0x563d6de11200 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c17400640 1 -- 172.21.15.94:0/3253709231 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c18030e50 con 0x563d6de11200 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c17400640 1 -- 172.21.15.94:0/3253709231 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f8c1c0d1ce0 con 0x563d6de11200 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 10 client.15423 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 20 client.15423 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de2c4d0 0x563d6de2e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 4 client.15423 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.247+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d6dd82b50 con 0x563d6de2c4d0 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.248+0000 7f8c31000640 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de2c4d0 0x563d6de2e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.248+0000 7f8c31000640 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de2c4d0 0x563d6de2e950 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1c2048b0a8759a server_cookie=c5d6c73c180ca6c4 in_seq=0 out_seq=0 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.248+0000 7f8c17400640 10 client.15423 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c17400640 1 -- 172.21.15.94:0/3253709231 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d6dd82b50 con 0x563d6de2c4d0 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c17400640 10 client.15423 handle_command_reply: tid=0 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 client.15423 shutdown 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de11200 msgr2=0x563d6de18f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de11200 0x563d6de18f00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c09b660 tx=0x7f8c1c097cd0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 shutdown_connections 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de2c4d0 0x563d6de2e950 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de12770 0x563d6de19440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 --2- 172.21.15.94:0/3253709231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de11200 0x563d6de18f00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c17400640 0 client.15423 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 >> 172.21.15.94:0/3253709231 conn(0x563d6ddd9f80 msgr2=0x563d6de13620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 shutdown_connections 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.249+0000 7f8c37252740 1 -- 172.21.15.94:0/3253709231 wait complete. 2024-06-03T01:13:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.250+0000 7f8c37252740 20 client.15423 trim_cache size 0 max 16384 2024-06-03T01:13:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 Processor -- start 2024-06-03T01:13:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 -- start start 2024-06-03T01:13:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de18060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de12770 0x563d6de195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de34260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c1b0d90 con 0x563d6de12770 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c1b2990 con 0x563d6de1bf80 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c37252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c1b1460 con 0x563d6de11200 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.266+0000 7f8c31000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de12770 0x563d6de195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c32400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de34260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de12770 0x563d6de195d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55750/0 (socket says 172.21.15.94:55750) 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c32400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de34260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45802/0 (socket says 172.21.15.94:45802) 2024-06-03T01:13:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31000640 1 -- 172.21.15.94:0/1710014404 learned_addr learned my addr 172.21.15.94:0/1710014404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31a00640 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de18060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31a00640 1 -- 172.21.15.94:0/1710014404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 msgr2=0x563d6de34260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31a00640 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de34260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31a00640 1 -- 172.21.15.94:0/1710014404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de12770 msgr2=0x563d6de195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31a00640 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de12770 0x563d6de195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31a00640 1 -- 172.21.15.94:0/1710014404 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6de11cd0 con 0x563d6de11200 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c31a00640 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de18060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0cd0e0 tx=0x7f8c1c099e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4424ff9fa181cf1a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c22000640 1 -- 172.21.15.94:0/1710014404 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c09b2c0 con 0x563d6de11200 2024-06-03T01:13:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c22000640 1 -- 172.21.15.94:0/1710014404 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c0a4520 con 0x563d6de11200 2024-06-03T01:13:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.267+0000 7f8c22000640 1 -- 172.21.15.94:0/1710014404 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00bc00 con 0x563d6de11200 2024-06-03T01:13:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 -- 172.21.15.94:0/1710014404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 msgr2=0x563d6de18060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de18060 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0cd0e0 tx=0x7f8c1c099e70 comp rx=0 tx=0).stop 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 -- 172.21.15.94:0/1710014404 shutdown_connections 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de1bf80 0x563d6de34260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de12770 0x563d6de195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 --2- 172.21.15.94:0/1710014404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de18060 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 -- 172.21.15.94:0/1710014404 >> 172.21.15.94:0/1710014404 conn(0x563d6ddd9f80 msgr2=0x563d6de18ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 -- 172.21.15.94:0/1710014404 shutdown_connections 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 -- 172.21.15.94:0/1710014404 wait complete. 2024-06-03T01:13:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.268+0000 7f8c37252740 1 -- start start 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 1 Processor -- start 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de24650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de1f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c1b2200 con 0x563d6de1bf80 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c1b1b30 con 0x563d6de12770 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c1aed70 con 0x563d6de11200 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c37252740 10 client.0 _ll_register_callbacks cb 0x7f8c1c00ef00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de24650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c31a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de24650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:47656/0 (socket says 172.21.15.94:47656) 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c31a00640 1 -- 172.21.15.94:0/2489345684 learned_addr learned my addr 172.21.15.94:0/2489345684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c31000640 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de1f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.269+0000 7f8c32400640 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c31a00640 1 -- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 msgr2=0x563d6de1f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c31a00640 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de1f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c31a00640 1 -- 172.21.15.94:0/2489345684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 msgr2=0x563d6de1ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c31a00640 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c31a00640 1 -- 172.21.15.94:0/2489345684 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6de204f0 con 0x563d6de11200 2024-06-03T01:13:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c31a00640 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de24650 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c000f80 tx=0x7f8c1c0cfa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354bfb4b498f004b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c17400640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c17400640 1 -- 172.21.15.94:0/2489345684 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c09b2c0 con 0x563d6de11200 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c17400640 1 -- 172.21.15.94:0/2489345684 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f8c1c0d5170 con 0x563d6de11200 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c1c0a4520 con 0x563d6de11200 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c17400640 1 -- 172.21.15.94:0/2489345684 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c1c081050 con 0x563d6de11200 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.270+0000 7f8c37252740 10 client.25567 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563d6de97020 con 0x563d6de11200 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c17400640 1 -- 172.21.15.94:0/2489345684 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c0a4520 con 0x563d6de11200 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c17400640 1 -- 172.21.15.94:0/2489345684 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f8c1c00b040 con 0x563d6de11200 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c37252740 10 client.25567 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c37252740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c37252740 20 client.25567 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c37252740 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de97740 0x563d6de99b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c37252740 4 client.25567 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:13:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d6dd7cea0 con 0x563d6de97740 2024-06-03T01:13:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.271+0000 7f8c31000640 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de97740 0x563d6de99b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.272+0000 7f8c31000640 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de97740 0x563d6de99b60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb15394890992b2 server_cookie=a5e28bbee10197e9 in_seq=0 out_seq=0 2024-06-03T01:13:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.272+0000 7f8c17400640 10 client.25567 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c17400640 1 -- 172.21.15.94:0/2489345684 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563d6dd7cea0 con 0x563d6de97740 2024-06-03T01:13:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c17400640 10 client.25567 handle_command_reply: tid=0 2024-06-03T01:13:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 client.25567 shutdown 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 msgr2=0x563d6de24650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de24650 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c000f80 tx=0x7f8c1c0cfa80 comp rx=0 tx=0).stop 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 shutdown_connections 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563d6de1bf80 0x563d6de1ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563d6de97740 0x563d6de99b60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563d6de12770 0x563d6de1f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 --2- 172.21.15.94:0/2489345684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d6de11200 0x563d6de24650 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c17400640 0 client.25567 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 >> 172.21.15.94:0/2489345684 conn(0x563d6ddd9f80 msgr2=0x563d6de12d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 shutdown_connections 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 1 -- 172.21.15.94:0/2489345684 wait complete. 2024-06-03T01:13:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.307+0000 7f8c37252740 20 client.25567 trim_cache size 0 max 16384 2024-06-03T01:13:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:13:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:13:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:13:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f8c100a2cf0 msgr2=0x7f8c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f8c100a2cf0 0x7f8c100a51b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c102770 tx=0x7f8c24005f90 comp rx=0 tx=0).stop 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c100590 msgr2=0x7f8c2c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c100590 0x7f8c2c19a3b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00efb0 tx=0x7f8c1c00c800 comp rx=0 tx=0).stop 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 shutdown_connections 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f8c100a2cf0 0x7f8c100a51b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c2c10e140 0x7f8c2c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c2c100f40 0x7f8c2c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 --2- 172.21.15.94:0/835056989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c2c100590 0x7f8c2c19a3b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 >> 172.21.15.94:0/835056989 conn(0x7f8c2c0fc1f0 msgr2=0x7f8c2c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 shutdown_connections 2024-06-03T01:13:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:29.308+0000 7f8c32e00640 1 -- 172.21.15.94:0/835056989 wait complete. 2024-06-03T01:13:30.065 INFO:tasks.workunit.client.0.smithi094.stdout: 1 57571 3.37 MB/sec warmup 97 sec latency 1135.311 ms 2024-06-03T01:13:30.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:29 smithi107 ceph-mon[36622]: pgmap v97: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 309 KiB/s wr, 61 op/s 2024-06-03T01:13:30.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:29 smithi094 ceph-mon[32119]: pgmap v97: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 309 KiB/s wr, 61 op/s 2024-06-03T01:13:30.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:29 smithi179 ceph-mon[37041]: pgmap v97: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 309 KiB/s wr, 61 op/s 2024-06-03T01:13:31.065 INFO:tasks.workunit.client.0.smithi094.stdout: 1 57583 3.34 MB/sec warmup 98 sec latency 2114.949 ms 2024-06-03T01:13:32.065 INFO:tasks.workunit.client.0.smithi094.stdout: 1 59215 3.37 MB/sec warmup 99 sec latency 14.838 ms 2024-06-03T01:13:32.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:31 smithi107 ceph-mon[36622]: pgmap v98: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 307 KiB/s wr, 59 op/s 2024-06-03T01:13:32.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:31 smithi094 ceph-mon[32119]: pgmap v98: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 307 KiB/s wr, 59 op/s 2024-06-03T01:13:32.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:31 smithi179 ceph-mon[37041]: pgmap v98: 129 pgs: 129 active+clean; 33 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 307 KiB/s wr, 59 op/s 2024-06-03T01:13:33.065 INFO:tasks.workunit.client.0.smithi094.stdout: 1 60572 3.46 MB/sec warmup 100 sec latency 16.892 ms 2024-06-03T01:13:33.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:32 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:13:33.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:32 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:13:33.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:32 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:13:34.066 INFO:tasks.workunit.client.0.smithi094.stdout: 1 61119 3.43 MB/sec warmup 101 sec latency 248.120 ms 2024-06-03T01:13:34.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:33 smithi107 ceph-mon[36622]: pgmap v99: 129 pgs: 129 active+clean; 36 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 656 KiB/s wr, 124 op/s 2024-06-03T01:13:34.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:33 smithi094 ceph-mon[32119]: pgmap v99: 129 pgs: 129 active+clean; 36 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 656 KiB/s wr, 124 op/s 2024-06-03T01:13:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:33 smithi179 ceph-mon[37041]: pgmap v99: 129 pgs: 129 active+clean; 36 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 656 KiB/s wr, 124 op/s 2024-06-03T01:13:35.065 INFO:tasks.workunit.client.0.smithi094.stdout: 1 61119 3.40 MB/sec warmup 102 sec latency 1248.245 ms 2024-06-03T01:13:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:34 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:35.366 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:34 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:13:35.366 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:34 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:13:35.367 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:34 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:35.367 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:34 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:13:35.367 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:34 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:13:35.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:34 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:13:35.368 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:34 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:13:35.368 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:34 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:13:36.066 INFO:tasks.workunit.client.0.smithi094.stdout: 1 61134 3.36 MB/sec warmup 103 sec latency 2228.133 ms 2024-06-03T01:13:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:35 smithi107 ceph-mon[36622]: pgmap v100: 129 pgs: 129 active+clean; 36 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 578 KiB/s wr, 106 op/s 2024-06-03T01:13:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:35 smithi094 ceph-mon[32119]: pgmap v100: 129 pgs: 129 active+clean; 36 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 578 KiB/s wr, 106 op/s 2024-06-03T01:13:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:35 smithi179 ceph-mon[37041]: pgmap v100: 129 pgs: 129 active+clean; 36 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 578 KiB/s wr, 106 op/s 2024-06-03T01:13:37.066 INFO:tasks.workunit.client.0.smithi094.stdout: 1 62771 3.40 MB/sec warmup 104 sec latency 14.716 ms 2024-06-03T01:13:38.067 INFO:tasks.workunit.client.0.smithi094.stdout: 1 64098 3.47 MB/sec warmup 105 sec latency 35.547 ms 2024-06-03T01:13:38.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:37 smithi107 ceph-mon[36622]: pgmap v101: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.0 MiB/s wr, 156 op/s 2024-06-03T01:13:38.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:37 smithi094 ceph-mon[32119]: pgmap v101: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.0 MiB/s wr, 156 op/s 2024-06-03T01:13:38.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:37 smithi179 ceph-mon[37041]: pgmap v101: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.0 MiB/s wr, 156 op/s 2024-06-03T01:13:39.067 INFO:tasks.workunit.client.0.smithi094.stdout: 1 64660 3.45 MB/sec warmup 106 sec latency 28.041 ms 2024-06-03T01:13:40.067 INFO:tasks.workunit.client.0.smithi094.stdout: 1 64660 3.42 MB/sec warmup 107 sec latency 1006.136 ms 2024-06-03T01:13:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:39 smithi107 ceph-mon[36622]: pgmap v102: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 810 KiB/s wr, 114 op/s 2024-06-03T01:13:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:39 smithi094 ceph-mon[32119]: pgmap v102: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 810 KiB/s wr, 114 op/s 2024-06-03T01:13:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:39 smithi179 ceph-mon[37041]: pgmap v102: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 810 KiB/s wr, 114 op/s 2024-06-03T01:13:41.067 INFO:tasks.workunit.client.0.smithi094.stdout: 1 64675 3.39 MB/sec warmup 108 sec latency 1984.871 ms 2024-06-03T01:13:42.067 INFO:tasks.workunit.client.0.smithi094.stdout: 1 66272 3.41 MB/sec warmup 109 sec latency 15.625 ms 2024-06-03T01:13:42.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:42 smithi107 ceph-mon[36622]: pgmap v103: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 810 KiB/s wr, 114 op/s 2024-06-03T01:13:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:42 smithi094 ceph-mon[32119]: pgmap v103: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 810 KiB/s wr, 114 op/s 2024-06-03T01:13:42.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:42 smithi179 ceph-mon[37041]: pgmap v103: 129 pgs: 129 active+clean; 38 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 810 KiB/s wr, 114 op/s 2024-06-03T01:13:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 -- 172.21.15.94:0/2292913094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d4073ca0 msgr2=0x7fc9d4074080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 --2- 172.21.15.94:0/2292913094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d4073ca0 0x7fc9d4074080 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009900 tx=0x7fc9c402a480 comp rx=0 tx=0).stop 2024-06-03T01:13:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 -- 172.21.15.94:0/2292913094 shutdown_connections 2024-06-03T01:13:42.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 --2- 172.21.15.94:0/2292913094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d410e290 0x7fc9d4110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 --2- 172.21.15.94:0/2292913094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d40745c0 0x7fc9d410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 --2- 172.21.15.94:0/2292913094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d4073ca0 0x7fc9d4074080 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 -- 172.21.15.94:0/2292913094 >> 172.21.15.94:0/2292913094 conn(0x7fc9d4078e30 msgr2=0x7fc9d4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 -- 172.21.15.94:0/2292913094 shutdown_connections 2024-06-03T01:13:42.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.458+0000 7fc9db800640 1 -- 172.21.15.94:0/2292913094 wait complete. 2024-06-03T01:13:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.459+0000 7fc9db800640 1 Processor -- start 2024-06-03T01:13:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.459+0000 7fc9db800640 1 -- start start 2024-06-03T01:13:42.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9db800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4073ca0 0x7fc9d419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9db800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d40745c0 0x7fc9d419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9db800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d410e290 0x7fc9d419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9db800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113fc0 con 0x7fc9d410e290 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9db800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113e40 con 0x7fc9d4073ca0 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9db800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4114140 con 0x7fc9d40745c0 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9d8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4073ca0 0x7fc9d419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d40745c0 0x7fc9d419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9d8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4073ca0 0x7fc9d419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41042/0 (socket says 172.21.15.94:41042) 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9d8e00640 1 -- 172.21.15.94:0/3963013494 learned_addr learned my addr 172.21.15.94:0/3963013494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9d9800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d410e290 0x7fc9d419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9cbe00640 1 -- 172.21.15.94:0/3963013494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4073ca0 msgr2=0x7fc9d419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4073ca0 0x7fc9d419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9cbe00640 1 -- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d410e290 msgr2=0x7fc9d419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d410e290 0x7fc9d419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9cbe00640 1 -- 172.21.15.94:0/3963013494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d419f350 con 0x7fc9d40745c0 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.460+0000 7fc9d9800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d410e290 0x7fc9d419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.461+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d40745c0 0x7fc9d419a8a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e9f0 tx=0x7fc9bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.461+0000 7fc9d2000640 1 -- 172.21.15.94:0/3963013494 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00ccc0 con 0x7fc9d40745c0 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.461+0000 7fc9d2000640 1 -- 172.21.15.94:0/3963013494 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9bc010910 con 0x7fc9d40745c0 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.461+0000 7fc9d2000640 1 -- 172.21.15.94:0/3963013494 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc018960 con 0x7fc9d40745c0 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.461+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d419f640 con 0x7fc9d40745c0 2024-06-03T01:13:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.461+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d41a6e80 con 0x7fc9d40745c0 2024-06-03T01:13:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.465+0000 7fc9de58a740 1 Processor -- start 2024-06-03T01:13:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.465+0000 7fc9de58a740 1 -- start start 2024-06-03T01:13:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.465+0000 7fc9d2000640 1 -- 172.21.15.94:0/3963013494 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc9bc010ac0 con 0x7fc9d40745c0 2024-06-03T01:13:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9de58a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fea5712420 con 0x55fea57ae040 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9de58a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fea56b9c50 con 0x55fea57af5b0 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9de58a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fea570e3e0 con 0x55fea57b8dc0 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41044/0 (socket says 172.21.15.94:41044) 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 -- 172.21.15.94:0/868805751 learned_addr learned my addr 172.21.15.94:0/868805751 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 -- 172.21.15.94:0/868805751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 msgr2=0x55fea57bb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57bb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 -- 172.21.15.94:0/868805751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 msgr2=0x55fea57ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.466+0000 7fc9d9800640 1 -- 172.21.15.94:0/868805751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea5426210 con 0x55fea57af5b0 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.467+0000 7fc9d9800640 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57adb00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009fb0 tx=0x7fc9cc011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb73fc01097e28c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.467+0000 7fc9ca000640 1 -- 172.21.15.94:0/868805751 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc012710 con 0x55fea57af5b0 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.467+0000 7fc9ca000640 1 -- 172.21.15.94:0/868805751 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc012d90 con 0x55fea57af5b0 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.467+0000 7fc9ca000640 1 -- 172.21.15.94:0/868805751 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc01a3f0 con 0x55fea57af5b0 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.467+0000 7fc9d2000640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc9b80a2e20 0x7fc9b80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.467+0000 7fc9d2000640 1 -- 172.21.15.94:0/3963013494 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fc9bc0cfb50 con 0x7fc9d40745c0 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc9b80a2e20 0x7fc9b80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 -- 172.21.15.94:0/868805751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 msgr2=0x55fea57adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57adb00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009fb0 tx=0x7fc9cc011870 comp rx=0 tx=0).stop 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 -- 172.21.15.94:0/868805751 shutdown_connections 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57bb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 --2- 172.21.15.94:0/868805751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57adb00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 -- 172.21.15.94:0/868805751 >> 172.21.15.94:0/868805751 conn(0x55fea5776f80 msgr2=0x55fea57aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 -- 172.21.15.94:0/868805751 shutdown_connections 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.468+0000 7fc9de58a740 1 -- 172.21.15.94:0/868805751 wait complete. 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.469+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc9b80a2e20 0x7fc9b80a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55fea57b0d10 tx=0x7fc9c4005a20 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.469+0000 7fc9de58a740 1 -- start start 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.469+0000 7fc9de58a740 1 Processor -- start 2024-06-03T01:13:42.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57ae040 0x55fea57b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57b8dc0 0x55fea57b6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9de58a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fea57bbb90 con 0x55fea57b8dc0 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9de58a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fea56cd0b0 con 0x55fea57ae040 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9de58a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fea57bbd10 con 0x55fea57af5b0 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9de58a740 10 client.0 _ll_register_callbacks cb 0x55fea56b8f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9d8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57ae040 0x55fea57b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49688/0 (socket says 172.21.15.94:49688) 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9d8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57ae040 0x55fea57b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41046/0 (socket says 172.21.15.94:41046) 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 -- 172.21.15.94:0/3901046673 learned_addr learned my addr 172.21.15.94:0/3901046673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:42.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9d9800640 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57b8dc0 0x55fea57b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 -- 172.21.15.94:0/3901046673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57ae040 msgr2=0x55fea57b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57ae040 0x55fea57b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 -- 172.21.15.94:0/3901046673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57b8dc0 msgr2=0x55fea57b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57b8dc0 0x55fea57b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.470+0000 7fc9cbe00640 1 -- 172.21.15.94:0/3901046673 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea54f2620 con 0x55fea57af5b0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57b6340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c6a0 tx=0x7fc9bc010ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d6282320719ea4e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9c2a00640 1 -- 172.21.15.94:0/3901046673 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc09b850 con 0x55fea57af5b0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9c2a00640 1 -- 172.21.15.94:0/3901046673 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fc9bc09b360 con 0x55fea57af5b0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9cc012d90 con 0x55fea57af5b0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9c2a00640 1 -- 172.21.15.94:0/3901046673 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc09b670 con 0x55fea57af5b0 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9de58a740 10 client.25579 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:42.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fea57c7f30 con 0x55fea57af5b0 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.471+0000 7fc9c2a00640 1 -- 172.21.15.94:0/3901046673 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc012d90 con 0x55fea57af5b0 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9c2a00640 1 -- 172.21.15.94:0/3901046673 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fc9bc018b00 con 0x55fea57af5b0 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9de58a740 10 client.25579 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9de58a740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9de58a740 20 client.25579 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9de58a740 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea57c93f0 0x55fea57cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9de58a740 4 client.25579 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fea571fb50 con 0x55fea57c93f0 2024-06-03T01:13:42.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.472+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea57c93f0 0x55fea57cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.473+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea57c93f0 0x55fea57cb850 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1538128e0c79 server_cookie=6850989365614862 in_seq=0 out_seq=0 2024-06-03T01:13:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.473+0000 7fc9c2a00640 10 client.25579 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.473+0000 7fc9c2a00640 1 -- 172.21.15.94:0/3901046673 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fea571fb50 con 0x55fea57c93f0 2024-06-03T01:13:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.473+0000 7fc9c2a00640 10 client.25579 handle_command_reply: tid=0 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.473+0000 7fc9de58a740 1 client.25579 shutdown 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.473+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 msgr2=0x55fea57b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.473+0000 7fc9de58a740 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57b6340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c6a0 tx=0x7fc9bc010ef0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 shutdown_connections 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57b8dc0 0x55fea57b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57b6340 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea57c93f0 0x55fea57cb850 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 --2- 172.21.15.94:0/3901046673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57ae040 0x55fea57b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 >> 172.21.15.94:0/3901046673 conn(0x55fea5776f80 msgr2=0x55fea57b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9c2a00640 0 client.25579 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 shutdown_connections 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 1 -- 172.21.15.94:0/3901046673 wait complete. 2024-06-03T01:13:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.474+0000 7fc9de58a740 20 client.25579 trim_cache size 0 max 16384 2024-06-03T01:13:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 Processor -- start 2024-06-03T01:13:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 -- start start 2024-06-03T01:13:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41b0e80 con 0x55fea57ae040 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41b2a80 con 0x55fea57af5b0 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.491+0000 7fc9de58a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41b1550 con 0x55fea57b8dc0 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9d9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:41062/0 (socket says 172.21.15.94:41062) 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 -- 172.21.15.94:0/2988958274 learned_addr learned my addr 172.21.15.94:0/2988958274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 -- 172.21.15.94:0/2988958274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 msgr2=0x55fea57d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 -- 172.21.15.94:0/2988958274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 msgr2=0x55fea57b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 -- 172.21.15.94:0/2988958274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea57a5dc0 con 0x55fea57af5b0 2024-06-03T01:13:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57d1160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc098170 tx=0x7fc9bc0d3270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f72083d112f534 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.492+0000 7fc9c9600640 1 -- 172.21.15.94:0/2988958274 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc09c410 con 0x55fea57af5b0 2024-06-03T01:13:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9c9600640 1 -- 172.21.15.94:0/2988958274 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9bc0d3390 con 0x55fea57af5b0 2024-06-03T01:13:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9c9600640 1 -- 172.21.15.94:0/2988958274 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc0a00c0 con 0x55fea57af5b0 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 -- 172.21.15.94:0/2988958274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 msgr2=0x55fea57d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57d1160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc098170 tx=0x7fc9bc0d3270 comp rx=0 tx=0).stop 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 -- 172.21.15.94:0/2988958274 shutdown_connections 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57b8dc0 0x55fea57d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57af5b0 0x55fea57d1160 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 --2- 172.21.15.94:0/2988958274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 -- 172.21.15.94:0/2988958274 >> 172.21.15.94:0/2988958274 conn(0x55fea5776f80 msgr2=0x55fea57b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.493+0000 7fc9de58a740 1 -- 172.21.15.94:0/2988958274 shutdown_connections 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.494+0000 7fc9de58a740 1 -- 172.21.15.94:0/2988958274 wait complete. 2024-06-03T01:13:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.494+0000 7fc9de58a740 1 -- start start 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.494+0000 7fc9de58a740 1 Processor -- start 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.494+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57bc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.494+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9de58a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57b8dc0 0x55fea5832120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9de58a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41b22f0 con 0x55fea57ae040 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9de58a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41b1c20 con 0x55fea57b8dc0 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9de58a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41aee60 con 0x55fea57af5b0 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9de58a740 10 client.0 _ll_register_callbacks cb 0x7fc9bc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57bc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49706/0 (socket says 172.21.15.94:49706) 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 -- 172.21.15.94:0/256943153 learned_addr learned my addr 172.21.15.94:0/256943153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57bc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9d9800640 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57b8dc0 0x55fea5832120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 -- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57b8dc0 msgr2=0x55fea5832120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57b8dc0 0x55fea5832120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 -- 172.21.15.94:0/256943153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 msgr2=0x55fea57bc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57bc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 -- 172.21.15.94:0/256943153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea57bce50 con 0x55fea57af5b0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9cbe00640 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57bc5c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0986e0 tx=0x7fc9bc097e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=712709622f1798a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.495+0000 7fc9c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9c2a00640 1 -- 172.21.15.94:0/256943153 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc081060 con 0x55fea57af5b0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9c2a00640 1 -- 172.21.15.94:0/256943153 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fc9bc09b160 con 0x55fea57af5b0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9bc0d3390 con 0x55fea57af5b0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9c2a00640 1 -- 172.21.15.94:0/256943153 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc0a4980 con 0x55fea57af5b0 2024-06-03T01:13:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 10 client.25585 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fea5833740 con 0x55fea57af5b0 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9c2a00640 1 -- 172.21.15.94:0/256943153 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9bc0d3390 con 0x55fea57af5b0 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9c2a00640 1 -- 172.21.15.94:0/256943153 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fc9bc0a4bd0 con 0x55fea57af5b0 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 10 client.25585 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 20 client.25585 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea5833e70 0x55fea5836290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 4 client.25585 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.496+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fea5719ea0 con 0x55fea5833e70 2024-06-03T01:13:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.497+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea5833e70 0x55fea5836290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.497+0000 7fc9c2a00640 10 client.25585 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.497+0000 7fc9d8e00640 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea5833e70 0x55fea5836290 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87084e3b4871241b server_cookie=f4e241a711033c99 in_seq=0 out_seq=0 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.551+0000 7fc9c2a00640 1 -- 172.21.15.94:0/256943153 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fea5719ea0 con 0x55fea5833e70 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.551+0000 7fc9c2a00640 10 client.25585 handle_command_reply: tid=0 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.551+0000 7fc9de58a740 1 client.25585 shutdown 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.551+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 msgr2=0x55fea57bc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.551+0000 7fc9de58a740 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57bc5c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0986e0 tx=0x7fc9bc097e40 comp rx=0 tx=0).stop 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 shutdown_connections 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fea5833e70 0x55fea5836290 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fea57b8dc0 0x55fea5832120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fea57af5b0 0x55fea57bc5c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 --2- 172.21.15.94:0/256943153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fea57ae040 0x55fea57bc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 >> 172.21.15.94:0/256943153 conn(0x55fea5776f80 msgr2=0x55fea57b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9c2a00640 0 client.25585 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 shutdown_connections 2024-06-03T01:13:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.552+0000 7fc9de58a740 1 -- 172.21.15.94:0/256943153 wait complete. 2024-06-03T01:13:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.553+0000 7fc9de58a740 20 client.25585 trim_cache size 0 max 16384 2024-06-03T01:13:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:13:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:13:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:13:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc9b80a2e20 msgr2=0x7fc9b80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc9b80a2e20 0x7fc9b80a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55fea57b0d10 tx=0x7fc9c4005a20 comp rx=0 tx=0).stop 2024-06-03T01:13:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d40745c0 msgr2=0x7fc9d419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d40745c0 0x7fc9d419a8a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e9f0 tx=0x7fc9bc00eeb0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 shutdown_connections 2024-06-03T01:13:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc9b80a2e20 0x7fc9b80a52e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc9d410e290 0x7fc9d419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9d40745c0 0x7fc9d419a8a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 --2- 172.21.15.94:0/3963013494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4073ca0 0x7fc9d419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.555+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 >> 172.21.15.94:0/3963013494 conn(0x7fc9d4078e30 msgr2=0x7fc9d40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.556+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 shutdown_connections 2024-06-03T01:13:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:42.556+0000 7fc9db800640 1 -- 172.21.15.94:0/3963013494 wait complete. 2024-06-03T01:13:43.067 INFO:tasks.workunit.client.0.smithi094.stdout: 1 67581 3.48 MB/sec warmup 110 sec latency 41.291 ms 2024-06-03T01:13:44.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 68165 3.47 MB/sec warmup 111 sec latency 19.123 ms 2024-06-03T01:13:44.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:44 smithi107 ceph-mon[36622]: pgmap v104: 129 pgs: 129 active+clean; 39 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-06-03T01:13:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:44 smithi094 ceph-mon[32119]: pgmap v104: 129 pgs: 129 active+clean; 39 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-06-03T01:13:44.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:44 smithi179 ceph-mon[37041]: pgmap v104: 129 pgs: 129 active+clean; 39 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-06-03T01:13:45.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 68211 3.43 MB/sec warmup 112 sec latency 951.192 ms 2024-06-03T01:13:46.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 68226 3.41 MB/sec warmup 113 sec latency 1932.574 ms 2024-06-03T01:13:46.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:46 smithi107 ceph-mon[36622]: pgmap v105: 129 pgs: 129 active+clean; 39 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 964 KiB/s wr, 111 op/s 2024-06-03T01:13:46.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:46 smithi094 ceph-mon[32119]: pgmap v105: 129 pgs: 129 active+clean; 39 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 964 KiB/s wr, 111 op/s 2024-06-03T01:13:46.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:46 smithi179 ceph-mon[37041]: pgmap v105: 129 pgs: 129 active+clean; 39 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 964 KiB/s wr, 111 op/s 2024-06-03T01:13:47.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 69827 3.43 MB/sec warmup 114 sec latency 15.220 ms 2024-06-03T01:13:48.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 71108 3.50 MB/sec warmup 115 sec latency 44.716 ms 2024-06-03T01:13:48.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:48 smithi107 ceph-mon[36622]: pgmap v106: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-03T01:13:48.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:48 smithi094 ceph-mon[32119]: pgmap v106: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-03T01:13:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:48 smithi179 ceph-mon[37041]: pgmap v106: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-06-03T01:13:49.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 71629 3.48 MB/sec warmup 116 sec latency 75.699 ms 2024-06-03T01:13:50.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 71629 3.45 MB/sec warmup 117 sec latency 1075.797 ms 2024-06-03T01:13:50.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:50 smithi107 ceph-mon[36622]: pgmap v107: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 995 KiB/s wr, 137 op/s 2024-06-03T01:13:50.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:50 smithi094 ceph-mon[32119]: pgmap v107: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 995 KiB/s wr, 137 op/s 2024-06-03T01:13:50.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:50 smithi179 ceph-mon[37041]: pgmap v107: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 995 KiB/s wr, 137 op/s 2024-06-03T01:13:51.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 71644 3.42 MB/sec warmup 118 sec latency 2054.741 ms 2024-06-03T01:13:52.068 INFO:tasks.workunit.client.0.smithi094.stdout: 1 72958 3.44 MB/sec warmup 119 sec latency 18.601 ms 2024-06-03T01:13:52.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:52 smithi107 ceph-mon[36622]: pgmap v108: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 995 KiB/s wr, 137 op/s 2024-06-03T01:13:52.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:52 smithi094 ceph-mon[32119]: pgmap v108: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 995 KiB/s wr, 137 op/s 2024-06-03T01:13:52.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:52 smithi179 ceph-mon[37041]: pgmap v108: 129 pgs: 129 active+clean; 37 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 995 KiB/s wr, 137 op/s 2024-06-03T01:13:54.069 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75110 1.71 MB/sec execute 1 sec latency 31.383 ms 2024-06-03T01:13:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:54 smithi107 ceph-mon[36622]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-06-03T01:13:54.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:54 smithi094 ceph-mon[32119]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-06-03T01:13:54.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:54 smithi179 ceph-mon[37041]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-06-03T01:13:55.069 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75210 0.90 MB/sec execute 2 sec latency 909.265 ms 2024-06-03T01:13:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 -- 172.21.15.94:0/87671024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f01077c0 msgr2=0x7f40f0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 --2- 172.21.15.94:0/87671024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f01077c0 0x7f40f0107ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009900 tx=0x7f40e002a410 comp rx=0 tx=0).stop 2024-06-03T01:13:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 -- 172.21.15.94:0/87671024 shutdown_connections 2024-06-03T01:13:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 --2- 172.21.15.94:0/87671024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f0102170 0x7f40f010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 --2- 172.21.15.94:0/87671024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40f01017b0 0x7f40f0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 --2- 172.21.15.94:0/87671024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f01077c0 0x7f40f0107ba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 -- 172.21.15.94:0/87671024 >> 172.21.15.94:0/87671024 conn(0x7f40f00fd2e0 msgr2=0x7f40f00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 -- 172.21.15.94:0/87671024 shutdown_connections 2024-06-03T01:13:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.708+0000 7f40efe00640 1 -- 172.21.15.94:0/87671024 wait complete. 2024-06-03T01:13:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.709+0000 7f40efe00640 1 Processor -- start 2024-06-03T01:13:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.709+0000 7f40efe00640 1 -- start start 2024-06-03T01:13:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40efe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40f01017b0 0x7f40f0073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f0102170 0x7f40f00741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f01077c0 0x7f40f0074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40efe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0111bd0 con 0x7f40f01077c0 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40efe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0111a50 con 0x7f40f0102170 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40efe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0111d50 con 0x7f40f01017b0 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40ee000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f0102170 0x7f40f00741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40ee000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f0102170 0x7f40f00741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:34856/0 (socket says 172.21.15.94:34856) 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40ee000640 1 -- 172.21.15.94:0/3872034484 learned_addr learned my addr 172.21.15.94:0/3872034484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40ef400640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f01077c0 0x7f40f0074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.710+0000 7f40eea00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40f01017b0 0x7f40f0073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40ee000640 1 -- 172.21.15.94:0/3872034484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40f01017b0 msgr2=0x7f40f0073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40ee000640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40f01017b0 0x7f40f0073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40ee000640 1 -- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f01077c0 msgr2=0x7f40f0074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40ee000640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f01077c0 0x7f40f0074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40ee000640 1 -- 172.21.15.94:0/3872034484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f01a6610 con 0x7f40f0102170 2024-06-03T01:13:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40ee000640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f0102170 0x7f40f00741e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00e9f0 tx=0x7f40dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40e7400640 1 -- 172.21.15.94:0/3872034484 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00cd80 con 0x7f40f0102170 2024-06-03T01:13:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f01a6900 con 0x7f40f0102170 2024-06-03T01:13:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40e7400640 1 -- 172.21.15.94:0/3872034484 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc005500 con 0x7f40f0102170 2024-06-03T01:13:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40e7400640 1 -- 172.21.15.94:0/3872034484 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc010690 con 0x7f40f0102170 2024-06-03T01:13:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.711+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f01a6e00 con 0x7f40f0102170 2024-06-03T01:13:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 Processor -- start 2024-06-03T01:13:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 -- start start 2024-06-03T01:13:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008cf770 0x55d8008cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008ce200 0x55d8008ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008db370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d800832420 con 0x55d8008cf770 2024-06-03T01:13:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8007d9c50 con 0x55d8008d8f80 2024-06-03T01:13:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40f8572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d80082e3e0 con 0x55d8008ce200 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40e7400640 1 -- 172.21.15.94:0/3872034484 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f40dc0046c0 con 0x7f40f0102170 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.715+0000 7f40ef400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008cf770 0x55d8008cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008cf770 0x55d8008cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58554/0 (socket says 172.21.15.94:58554) 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 -- 172.21.15.94:0/3850760310 learned_addr learned my addr 172.21.15.94:0/3850760310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40eea00640 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008ce200 0x55d8008ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ee000640 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008db370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 -- 172.21.15.94:0/3850760310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008ce200 msgr2=0x55d8008ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008ce200 0x55d8008ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 -- 172.21.15.94:0/3850760310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 msgr2=0x55d8008db370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008db370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 -- 172.21.15.94:0/3850760310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d800546210 con 0x55d8008cf770 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.716+0000 7f40ef400640 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008cf770 0x55d8008cdcc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009f50 tx=0x7f40e80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d21f40a4bb350aa server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40dbe00640 1 -- 172.21.15.94:0/3850760310 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8004d20 con 0x55d8008cf770 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40dbe00640 1 -- 172.21.15.94:0/3850760310 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8004050 con 0x55d8008cf770 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40dbe00640 1 -- 172.21.15.94:0/3850760310 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8005260 con 0x55d8008cf770 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40e7400640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f40d40a2cf0 0x7f40d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40e7400640 1 -- 172.21.15.94:0/3872034484 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f40dc0cfb90 con 0x7f40f0102170 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40f8572740 1 -- 172.21.15.94:0/3850760310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008cf770 msgr2=0x55d8008cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40eea00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f40d40a2cf0 0x7f40d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.717+0000 7f40f8572740 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008cf770 0x55d8008cdcc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009f50 tx=0x7f40e80118d0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 -- 172.21.15.94:0/3850760310 shutdown_connections 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008db370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008ce200 0x55d8008ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 --2- 172.21.15.94:0/3850760310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008cf770 0x55d8008cdcc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 -- 172.21.15.94:0/3850760310 >> 172.21.15.94:0/3850760310 conn(0x55d800896f80 msgr2=0x55d8008ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 -- 172.21.15.94:0/3850760310 shutdown_connections 2024-06-03T01:13:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 -- 172.21.15.94:0/3850760310 wait complete. 2024-06-03T01:13:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40eea00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f40d40a2cf0 0x7f40d40a51b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009230 tx=0x7f40e00058d0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 -- start start 2024-06-03T01:13:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.718+0000 7f40f8572740 1 Processor -- start 2024-06-03T01:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40f8572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008ce200 0x55d8008d5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40f8572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40f8572740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40f8572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d8008dbd50 con 0x55d8008ce200 2024-06-03T01:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40f8572740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8007ed0b0 con 0x55d8008d8f80 2024-06-03T01:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40f8572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8008dbed0 con 0x55d8008cf770 2024-06-03T01:13:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40f8572740 10 client.0 _ll_register_callbacks cb 0x55d8007d8f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40eea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008ce200 0x55d8008d5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40eea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008ce200 0x55d8008d5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58564/0 (socket says 172.21.15.94:58564) 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40eea00640 1 -- 172.21.15.94:0/2480175089 learned_addr learned my addr 172.21.15.94:0/2480175089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40ef400640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.719+0000 7f40ee000640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40eea00640 1 -- 172.21.15.94:0/2480175089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 msgr2=0x55d8008d64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40eea00640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008d64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40eea00640 1 -- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 msgr2=0x55d8008d6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40eea00640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008d6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40eea00640 1 -- 172.21.15.94:0/2480175089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d800612620 con 0x55d8008ce200 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40ee000640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008d64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40eea00640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008ce200 0x55d8008d5f90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40e0033000 tx=0x7f40e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48749dee1e87c1b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40d8c00640 1 -- 172.21.15.94:0/2480175089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0035040 con 0x55d8008ce200 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40d8c00640 1 -- 172.21.15.94:0/2480175089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f40e0003dc0 con 0x55d8008ce200 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.720+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e8004050 con 0x55d8008ce200 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40d8c00640 1 -- 172.21.15.94:0/2480175089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f40e0031540 con 0x55d8008ce200 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 10 client.15459 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d8008e80c0 con 0x55d8008ce200 2024-06-03T01:13:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40d8c00640 1 -- 172.21.15.94:0/2480175089 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8004050 con 0x55d8008ce200 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40d8c00640 1 -- 172.21.15.94:0/2480175089 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f40e0044030 con 0x55d8008ce200 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 10 client.15459 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 20 client.15459 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d8008e95d0 0x55d8008eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 4 client.15459 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.721+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d80083fb50 con 0x55d8008e95d0 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.722+0000 7f40ee000640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d8008e95d0 0x55d8008eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.722+0000 7f40ee000640 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d8008e95d0 0x55d8008eba20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57aa89e71bde724b server_cookie=b342289f2bebe006 in_seq=0 out_seq=0 2024-06-03T01:13:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.722+0000 7f40d8c00640 10 client.15459 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40d8c00640 1 -- 172.21.15.94:0/2480175089 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d80083fb50 con 0x55d8008e95d0 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40d8c00640 10 client.15459 handle_command_reply: tid=0 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 client.15459 shutdown 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008ce200 msgr2=0x55d8008d5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008ce200 0x55d8008d5f90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40e0033000 tx=0x7f40e0009260 comp rx=0 tx=0).stop 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 shutdown_connections 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d8008e95d0 0x55d8008eba20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008d8f80 0x55d8008d6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008d64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40d8c00640 0 client.15459 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 --2- 172.21.15.94:0/2480175089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008ce200 0x55d8008d5f90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 >> 172.21.15.94:0/2480175089 conn(0x55d800896f80 msgr2=0x55d8008d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 shutdown_connections 2024-06-03T01:13:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 1 -- 172.21.15.94:0/2480175089 wait complete. 2024-06-03T01:13:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.724+0000 7f40f8572740 20 client.15459 trim_cache size 0 max 16384 2024-06-03T01:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.740+0000 7f40f8572740 1 Processor -- start 2024-06-03T01:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.740+0000 7f40f8572740 1 -- start start 2024-06-03T01:13:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.740+0000 7f40f8572740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008f1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.740+0000 7f40f8572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40f8572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008d5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40f8572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01b0d10 con 0x55d8008d8f80 2024-06-03T01:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40f8572740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01b2910 con 0x55d8008ce200 2024-06-03T01:13:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40f8572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01b13e0 con 0x55d8008cf770 2024-06-03T01:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008d5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008d5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58572/0 (socket says 172.21.15.94:58572) 2024-06-03T01:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 -- 172.21.15.94:0/154198088 learned_addr learned my addr 172.21.15.94:0/154198088 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ee000640 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40eea00640 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 -- 172.21.15.94:0/154198088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 msgr2=0x55d8008f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 -- 172.21.15.94:0/154198088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 msgr2=0x55d8008f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.741+0000 7f40ef400640 1 -- 172.21.15.94:0/154198088 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8008cf380 con 0x55d8008d8f80 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.742+0000 7f40ef400640 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008d5f50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40e8000f80 tx=0x7f40e8003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e314db2d5035ef4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.742+0000 7f40db400640 1 -- 172.21.15.94:0/154198088 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80052b0 con 0x55d8008d8f80 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.742+0000 7f40db400640 1 -- 172.21.15.94:0/154198088 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8005bd0 con 0x55d8008d8f80 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.742+0000 7f40db400640 1 -- 172.21.15.94:0/154198088 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800fbf0 con 0x55d8008d8f80 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 -- 172.21.15.94:0/154198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 msgr2=0x55d8008d5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008d5f50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40e8000f80 tx=0x7f40e8003c90 comp rx=0 tx=0).stop 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 -- 172.21.15.94:0/154198088 shutdown_connections 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008d5f50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 --2- 172.21.15.94:0/154198088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 -- 172.21.15.94:0/154198088 >> 172.21.15.94:0/154198088 conn(0x55d800896f80 msgr2=0x55d8008d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 -- 172.21.15.94:0/154198088 shutdown_connections 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 -- 172.21.15.94:0/154198088 wait complete. 2024-06-03T01:13:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 -- start start 2024-06-03T01:13:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.743+0000 7f40f8572740 1 Processor -- start 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40f8572740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008e1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40f8572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008e1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40f8572740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008dc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40f8572740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01b2180 con 0x55d8008d8f80 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40f8572740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01b1ab0 con 0x55d8008ce200 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40f8572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01aecf0 con 0x55d8008cf770 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40f8572740 10 client.0 _ll_register_callbacks cb 0x7f40e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:13:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40ef400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008dc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40eea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40eea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008e1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:34890/0 (socket says 172.21.15.94:34890) 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40ef400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008dc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58576/0 (socket says 172.21.15.94:58576) 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40eea00640 1 -- 172.21.15.94:0/3074050170 learned_addr learned my addr 172.21.15.94:0/3074050170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.744+0000 7f40ee000640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008e1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40ef400640 1 -- 172.21.15.94:0/3074050170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 msgr2=0x55d8008e1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40ef400640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008e1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40ef400640 1 -- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 msgr2=0x55d8008e1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40ef400640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008e1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40ef400640 1 -- 172.21.15.94:0/3074050170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8008dd0d0 con 0x55d8008d8f80 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40eea00640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40ee000640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008e1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40ef400640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008dc8a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f40e801b040 tx=0x7f40e8003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68da6d1b14fa0e9a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40d8c00640 1 -- 172.21.15.94:0/3074050170 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800f480 con 0x55d8008d8f80 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40d8c00640 1 -- 172.21.15.94:0/3074050170 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f40e800f9e0 con 0x55d8008d8f80 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.745+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e8005bd0 con 0x55d8008d8f80 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40d8c00640 1 -- 172.21.15.94:0/3074050170 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f40e803b050 con 0x55d8008d8f80 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 10 client.15465 fetch_fsmap learned FSMap version 32 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d800953e90 con 0x55d8008d8f80 2024-06-03T01:13:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40d8c00640 1 -- 172.21.15.94:0/3074050170 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8005bd0 con 0x55d8008d8f80 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40d8c00640 1 -- 172.21.15.94:0/3074050170 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7f40e802c430 con 0x55d8008d8f80 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 10 client.15465 fetch_fsmap finished waiting for FSMap version 32 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 20 client.15465 populate_metadata read hostname 'smithi094' 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d800954340 0x55d800956760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 4 client.15465 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d800839ea0 con 0x55d800954340 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.746+0000 7f40eea00640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d800954340 0x55d800956760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.747+0000 7f40d8c00640 10 client.15465 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.747+0000 7f40eea00640 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d800954340 0x55d800956760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8c680f4fd7677c server_cookie=57fa960e293a2c89 in_seq=0 out_seq=0 2024-06-03T01:13:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40d8c00640 1 -- 172.21.15.94:0/3074050170 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d800839ea0 con 0x55d800954340 2024-06-03T01:13:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40d8c00640 10 client.15465 handle_command_reply: tid=0 2024-06-03T01:13:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 client.15465 shutdown 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 msgr2=0x55d8008dc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008dc8a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f40e801b040 tx=0x7f40e8003a30 comp rx=0 tx=0).stop 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 shutdown_connections 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8008d8f80 0x55d8008dc8a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8008cf770 0x55d8008e1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d800954340 0x55d800956760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 --2- 172.21.15.94:0/3074050170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8008ce200 0x55d8008e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40d8c00640 0 client.15465 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 >> 172.21.15.94:0/3074050170 conn(0x55d800896f80 msgr2=0x55d8008c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 shutdown_connections 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 1 -- 172.21.15.94:0/3074050170 wait complete. 2024-06-03T01:13:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.790+0000 7f40f8572740 20 client.15465 trim_cache size 0 max 16384 2024-06-03T01:13:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:13:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:13:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f40d40a2cf0 msgr2=0x7f40d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f40d40a2cf0 0x7f40d40a51b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009230 tx=0x7f40e00058d0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f0102170 msgr2=0x7f40f00741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f0102170 0x7f40f00741e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00e9f0 tx=0x7f40dc00edf0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 shutdown_connections 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f40d40a2cf0 0x7f40d40a51b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40f01077c0 0x7f40f0074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40f0102170 0x7f40f00741e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 --2- 172.21.15.94:0/3872034484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40f01017b0 0x7f40f0073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:13:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 >> 172.21.15.94:0/3872034484 conn(0x7f40f00fd2e0 msgr2=0x7f40f00fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:13:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 shutdown_connections 2024-06-03T01:13:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:13:55.791+0000 7f40efe00640 1 -- 172.21.15.94:0/3872034484 wait complete. 2024-06-03T01:13:56.069 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75222 0.63 MB/sec execute 3 sec latency 1891.845 ms 2024-06-03T01:13:56.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:56 smithi094 ceph-mon[32119]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 939 KiB/s wr, 123 op/s 2024-06-03T01:13:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:56 smithi179 ceph-mon[37041]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 939 KiB/s wr, 123 op/s 2024-06-03T01:13:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:56 smithi107 ceph-mon[36622]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 939 KiB/s wr, 123 op/s 2024-06-03T01:13:57.069 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75266 0.49 MB/sec execute 4 sec latency 935.022 ms 2024-06-03T01:13:58.069 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75266 0.39 MB/sec execute 5 sec latency 1935.186 ms 2024-06-03T01:13:58.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:13:58 smithi094 ceph-mon[32119]: pgmap v111: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-06-03T01:13:58.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:13:58 smithi179 ceph-mon[37041]: pgmap v111: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-06-03T01:13:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:13:58 smithi107 ceph-mon[36622]: pgmap v111: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-06-03T01:13:59.069 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75266 0.32 MB/sec execute 6 sec latency 2935.319 ms 2024-06-03T01:14:00.069 INFO:tasks.workunit.client.0.smithi094.stdout: 1 75266 0.28 MB/sec execute 7 sec latency 3935.513 ms 2024-06-03T01:14:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:00 smithi107 ceph-mon[36622]: pgmap v112: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 759 KiB/s wr, 109 op/s 2024-06-03T01:14:00.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:00 smithi094 ceph-mon[32119]: pgmap v112: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 759 KiB/s wr, 109 op/s 2024-06-03T01:14:00.854 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:00 smithi179 ceph-mon[37041]: pgmap v112: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 759 KiB/s wr, 109 op/s 2024-06-03T01:14:01.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 76247 0.87 MB/sec execute 8 sec latency 4080.202 ms 2024-06-03T01:14:02.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 77787 1.72 MB/sec execute 9 sec latency 15.650 ms 2024-06-03T01:14:02.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:02 smithi107 ceph-mon[36622]: pgmap v113: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 759 KiB/s wr, 109 op/s 2024-06-03T01:14:02.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:02 smithi094 ceph-mon[32119]: pgmap v113: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 759 KiB/s wr, 109 op/s 2024-06-03T01:14:02.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:02 smithi179 ceph-mon[37041]: pgmap v113: 129 pgs: 129 active+clean; 40 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 759 KiB/s wr, 109 op/s 2024-06-03T01:14:03.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 78544 2.08 MB/sec execute 10 sec latency 17.768 ms 2024-06-03T01:14:04.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 78877 1.93 MB/sec execute 11 sec latency 627.256 ms 2024-06-03T01:14:04.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:04 smithi107 ceph-mon[36622]: pgmap v114: 129 pgs: 129 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-03T01:14:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:04 smithi094 ceph-mon[32119]: pgmap v114: 129 pgs: 129 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-03T01:14:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:04 smithi179 ceph-mon[37041]: pgmap v114: 129 pgs: 129 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-03T01:14:05.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 78877 1.77 MB/sec execute 12 sec latency 1627.399 ms 2024-06-03T01:14:06.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 78896 1.64 MB/sec execute 13 sec latency 2600.826 ms 2024-06-03T01:14:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:06 smithi094 ceph-mon[32119]: pgmap v115: 129 pgs: 129 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 863 KiB/s wr, 114 op/s 2024-06-03T01:14:06.854 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:06 smithi179 ceph-mon[37041]: pgmap v115: 129 pgs: 129 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 863 KiB/s wr, 114 op/s 2024-06-03T01:14:07.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:06 smithi107 ceph-mon[36622]: pgmap v115: 129 pgs: 129 active+clean; 41 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 863 KiB/s wr, 114 op/s 2024-06-03T01:14:07.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 80645 2.05 MB/sec execute 14 sec latency 12.311 ms 2024-06-03T01:14:07.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:07 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:07.854 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:07 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:08.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:07 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:08.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 81971 2.66 MB/sec execute 15 sec latency 39.836 ms 2024-06-03T01:14:08.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:08 smithi094 ceph-mon[32119]: pgmap v116: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-03T01:14:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:08 smithi179 ceph-mon[37041]: pgmap v116: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 -- 172.21.15.94:0/3441778277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4678103570 msgr2=0x7f467810ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 --2- 172.21.15.94:0/3441778277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4678103570 0x7f467810ba50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4664009860 tx=0x7f466402a3d0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 -- 172.21.15.94:0/3441778277 shutdown_connections 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 --2- 172.21.15.94:0/3441778277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467810c020 0x7f467810e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 --2- 172.21.15.94:0/3441778277 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4678103570 0x7f467810ba50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 --2- 172.21.15.94:0/3441778277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4678102c50 0x7f4678103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 -- 172.21.15.94:0/3441778277 >> 172.21.15.94:0/3441778277 conn(0x7f4678076c20 msgr2=0x7f4678077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 -- 172.21.15.94:0/3441778277 shutdown_connections 2024-06-03T01:14:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.932+0000 7f467d200640 1 -- 172.21.15.94:0/3441778277 wait complete. 2024-06-03T01:14:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.933+0000 7f467d200640 1 Processor -- start 2024-06-03T01:14:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f467d200640 1 -- start start 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f467d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4678102c50 0x7f4678198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f467d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4678103570 0x7f4678198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f467d200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467810c020 0x7f467819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f467d200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111db0 con 0x7f467810c020 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f467d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111c30 con 0x7f4678102c50 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f467d200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111f30 con 0x7f4678103570 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f4675e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4678102c50 0x7f4678198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f4676800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467810c020 0x7f467819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f4675e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4678102c50 0x7f4678198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:39896/0 (socket says 172.21.15.94:39896) 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f4675e00640 1 -- 172.21.15.94:0/1871765521 learned_addr learned my addr 172.21.15.94:0/1871765521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.934+0000 7f4675400640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4678103570 0x7f4678198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f4675e00640 1 -- 172.21.15.94:0/1871765521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4678103570 msgr2=0x7f4678198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f4675e00640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4678103570 0x7f4678198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f4675e00640 1 -- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467810c020 msgr2=0x7f467819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f4675e00640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467810c020 0x7f467819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f4675e00640 1 -- 172.21.15.94:0/1871765521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467819d200 con 0x7f4678102c50 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f4675e00640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4678102c50 0x7f4678198210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f466800e9f0 tx=0x7f466800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f466ea00640 1 -- 172.21.15.94:0/1871765521 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466800cd80 con 0x7f4678102c50 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467819d4f0 con 0x7f4678102c50 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f466ea00640 1 -- 172.21.15.94:0/1871765521 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4668005500 con 0x7f4678102c50 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f466ea00640 1 -- 172.21.15.94:0/1871765521 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668010690 con 0x7f4678102c50 2024-06-03T01:14:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.935+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781a4d80 con 0x7f4678102c50 2024-06-03T01:14:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.938+0000 7f46801aa740 1 Processor -- start 2024-06-03T01:14:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f46801aa740 1 -- start start 2024-06-03T01:14:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f466ea00640 1 -- 172.21.15.94:0/1871765521 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f46680046c0 con 0x7f4678102c50 2024-06-03T01:14:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f46801aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f46801aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e214c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f46801aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f46801aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564908d85420 con 0x564908e21040 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f46801aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564908d2cc50 con 0x564908e225b0 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f46801aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564908d813e0 con 0x564908e301d0 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f4676800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f4675e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f4676800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e20b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:39912/0 (socket says 172.21.15.94:39912) 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f4676800640 1 -- 172.21.15.94:0/2215647726 learned_addr learned my addr 172.21.15.94:0/2215647726 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.939+0000 7f4675400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4676800640 1 -- 172.21.15.94:0/2215647726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 msgr2=0x564908e325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4676800640 1 --2- 172.21.15.94:0/2215647726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4676800640 1 -- 172.21.15.94:0/2215647726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 msgr2=0x564908e214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4676800640 1 --2- 172.21.15.94:0/2215647726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4676800640 1 -- 172.21.15.94:0/2215647726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564908a99210 con 0x564908e225b0 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4676800640 1 --2- 172.21.15.94:0/2215647726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e20b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4670009a10 tx=0x7f46700095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4469908ccb7cfae2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4663400640 1 -- 172.21.15.94:0/2215647726 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670003970 con 0x564908e225b0 2024-06-03T01:14:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4663400640 1 -- 172.21.15.94:0/2215647726 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467000d6b0 con 0x564908e225b0 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f4663400640 1 -- 172.21.15.94:0/2215647726 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670004360 con 0x564908e225b0 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.940+0000 7f466ea00640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f465c0a2da0 0x7f465c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f466ea00640 1 -- 172.21.15.94:0/1871765521 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f46680cfc30 con 0x7f4678102c50 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f4675e00640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f465c0a2da0 0x7f465c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 -- 172.21.15.94:0/2215647726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 msgr2=0x564908e20b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 --2- 172.21.15.94:0/2215647726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e20b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4670009a10 tx=0x7f46700095c0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 -- 172.21.15.94:0/2215647726 shutdown_connections 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 --2- 172.21.15.94:0/2215647726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 --2- 172.21.15.94:0/2215647726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 --2- 172.21.15.94:0/2215647726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e20b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 -- 172.21.15.94:0/2215647726 >> 172.21.15.94:0/2215647726 conn(0x564908de9f80 msgr2=0x564908e1d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 -- 172.21.15.94:0/2215647726 shutdown_connections 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f46801aa740 1 -- 172.21.15.94:0/2215647726 wait complete. 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.941+0000 7f4675e00640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f465c0a2da0 0x7f465c0a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x564908e23d10 tx=0x7f46680d36e0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 -- start start 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 Processor -- start 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e2d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e2d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564908e32fa0 con 0x564908e21040 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564908d400b0 con 0x564908e225b0 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564908e33120 con 0x564908e301d0 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.942+0000 7f46801aa740 10 client.0 _ll_register_callbacks cb 0x564908d2bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4676800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e2d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4676800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e2d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60908/0 (socket says 172.21.15.94:60908) 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4676800640 1 -- 172.21.15.94:0/2784101292 learned_addr learned my addr 172.21.15.94:0/2784101292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4675400640 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e2d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4676800640 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4675400640 1 -- 172.21.15.94:0/2784101292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 msgr2=0x564908e2dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4675400640 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e2dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4675400640 1 -- 172.21.15.94:0/2784101292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 msgr2=0x564908e2d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4675400640 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e2d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4675400640 1 -- 172.21.15.94:0/2784101292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564908b65620 con 0x564908e225b0 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.943+0000 7f4675400640 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e2d780 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4664005e00 tx=0x7f4664032ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f062b92e81501ba7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f4657e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f4657e00640 1 -- 172.21.15.94:0/2784101292 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640305b0 con 0x564908e225b0 2024-06-03T01:14:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f4657e00640 1 -- 172.21.15.94:0/2784101292 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f4664030750 con 0x564908e225b0 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467000d6b0 con 0x564908e225b0 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f4657e00640 1 -- 172.21.15.94:0/2784101292 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f466404f050 con 0x564908e225b0 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f46801aa740 10 client.25601 fetch_fsmap learned FSMap version 33 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564908e3f360 con 0x564908e225b0 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.944+0000 7f4657e00640 1 -- 172.21.15.94:0/2784101292 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467000d6b0 con 0x564908e225b0 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f4657e00640 1 -- 172.21.15.94:0/2784101292 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f4664004c40 con 0x564908e225b0 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f46801aa740 10 client.25601 fetch_fsmap finished waiting for FSMap version 33 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f46801aa740 10 client.25601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f46801aa740 20 client.25601 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f46801aa740 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e40870 0x564908e42cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f46801aa740 4 client.25601 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564908d92b50 con 0x564908e40870 2024-06-03T01:14:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.945+0000 7f4676800640 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e40870 0x564908e42cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.946+0000 7f4676800640 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e40870 0x564908e42cc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597deda34f02be28 server_cookie=a881f994b84bdc76 in_seq=0 out_seq=0 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.946+0000 7f4657e00640 10 client.25601 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.946+0000 7f4657e00640 1 -- 172.21.15.94:0/2784101292 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564908d92b50 con 0x564908e40870 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.946+0000 7f4657e00640 10 client.25601 handle_command_reply: tid=0 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 client.25601 shutdown 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 msgr2=0x564908e2d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e2d780 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4664005e00 tx=0x7f4664032ed0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 shutdown_connections 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e301d0 0x564908e2dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e40870 0x564908e42cc0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e225b0 0x564908e2d780 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 --2- 172.21.15.94:0/2784101292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908e21040 0x564908e2d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f4657e00640 0 client.25601 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 >> 172.21.15.94:0/2784101292 conn(0x564908de9f80 msgr2=0x564908e234a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 shutdown_connections 2024-06-03T01:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 1 -- 172.21.15.94:0/2784101292 wait complete. 2024-06-03T01:14:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.947+0000 7f46801aa740 20 client.25601 trim_cache size 0 max 16384 2024-06-03T01:14:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 Processor -- start 2024-06-03T01:14:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 -- start start 2024-06-03T01:14:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e1d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e2c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e1c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46781acaa0 con 0x564908de9f80 2024-06-03T01:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46781ae6d0 con 0x564908e21040 2024-06-03T01:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.964+0000 7f46801aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46781ad170 con 0x564908e225b0 2024-06-03T01:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e1d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e1d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60910/0 (socket says 172.21.15.94:60910) 2024-06-03T01:14:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 -- 172.21.15.94:0/3622415559 learned_addr learned my addr 172.21.15.94:0/3622415559 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4675400640 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e2c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e1c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 -- 172.21.15.94:0/3622415559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 msgr2=0x564908e1c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e1c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 -- 172.21.15.94:0/3622415559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 msgr2=0x564908e2c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e2c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 -- 172.21.15.94:0/3622415559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564908e3c240 con 0x564908de9f80 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.965+0000 7f4676800640 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e1d160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4670009c50 tx=0x7f4670005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6639ef22b21f7893 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f4662a00640 1 -- 172.21.15.94:0/3622415559 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467000d770 con 0x564908de9f80 2024-06-03T01:14:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f4662a00640 1 -- 172.21.15.94:0/3622415559 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467000ddf0 con 0x564908de9f80 2024-06-03T01:14:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f4662a00640 1 -- 172.21.15.94:0/3622415559 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700312c0 con 0x564908de9f80 2024-06-03T01:14:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 -- 172.21.15.94:0/3622415559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 msgr2=0x564908e1d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e1d160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4670009c50 tx=0x7f4670005a50 comp rx=0 tx=0).stop 2024-06-03T01:14:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 -- 172.21.15.94:0/3622415559 shutdown_connections 2024-06-03T01:14:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e1c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e2c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 --2- 172.21.15.94:0/3622415559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e1d160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 -- 172.21.15.94:0/3622415559 >> 172.21.15.94:0/3622415559 conn(0x564908e28c10 msgr2=0x564908e1bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.966+0000 7f46801aa740 1 -- 172.21.15.94:0/3622415559 shutdown_connections 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.967+0000 7f46801aa740 1 -- 172.21.15.94:0/3622415559 wait complete. 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.967+0000 7f46801aa740 1 -- start start 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.967+0000 7f46801aa740 1 Processor -- start 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f46801aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e38a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f46801aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f46801aa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e342d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f46801aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46781adf40 con 0x564908de9f80 2024-06-03T01:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f46801aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46781ad840 con 0x564908e21040 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f46801aa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46781aaa80 con 0x564908e225b0 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f46801aa740 10 client.0 _ll_register_callbacks cb 0x7f46640096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e38a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e38a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60914/0 (socket says 172.21.15.94:60914) 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 -- 172.21.15.94:0/1154602782 learned_addr learned my addr 172.21.15.94:0/1154602782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4675400640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 -- 172.21.15.94:0/1154602782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 msgr2=0x564908e342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e342d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 -- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 msgr2=0x564908e33d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e33d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4676800640 1 -- 172.21.15.94:0/1154602782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564908e349c0 con 0x564908de9f80 2024-06-03T01:14:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.968+0000 7f4675400640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e33d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f4676800640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e38a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4670003a40 tx=0x7f4670005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0287ca4a37ef6ef server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f4657e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:14:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f4657e00640 1 -- 172.21.15.94:0/1154602782 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700050f0 con 0x564908de9f80 2024-06-03T01:14:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f4657e00640 1 -- 172.21.15.94:0/1154602782 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f4670005290 con 0x564908de9f80 2024-06-03T01:14:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467000ddf0 con 0x564908de9f80 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f4657e00640 1 -- 172.21.15.94:0/1154602782 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f467003e050 con 0x564908de9f80 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f46801aa740 10 client.15483 fetch_fsmap learned FSMap version 33 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564908eab470 con 0x564908de9f80 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.969+0000 7f4657e00640 1 -- 172.21.15.94:0/1154602782 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467000ddf0 con 0x564908de9f80 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f4657e00640 1 -- 172.21.15.94:0/1154602782 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f467000b040 con 0x564908de9f80 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f46801aa740 10 client.15483 fetch_fsmap finished waiting for FSMap version 33 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f46801aa740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f46801aa740 20 client.15483 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f46801aa740 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e301d0 0x564908ead930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f46801aa740 4 client.15483 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564908d8cea0 con 0x564908e301d0 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.970+0000 7f4675400640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e301d0 0x564908ead930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.971+0000 7f4675400640 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e301d0 0x564908ead930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317e0d384f3471e6 server_cookie=4535d0d48b6c6884 in_seq=0 out_seq=0 2024-06-03T01:14:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:08.971+0000 7f4657e00640 10 client.15483 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f4657e00640 1 -- 172.21.15.94:0/1154602782 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564908d8cea0 con 0x564908e301d0 2024-06-03T01:14:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f4657e00640 10 client.15483 handle_command_reply: tid=0 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 client.15483 shutdown 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 msgr2=0x564908e38a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e38a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4670003a40 tx=0x7f4670005680 comp rx=0 tx=0).stop 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 shutdown_connections 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564908e225b0 0x564908e342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564908e301d0 0x564908ead930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564908e21040 0x564908e33d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 --2- 172.21.15.94:0/1154602782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564908de9f80 0x564908e38a20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f4657e00640 0 client.15483 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 >> 172.21.15.94:0/1154602782 conn(0x564908e28c10 msgr2=0x564908e2d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 shutdown_connections 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.008+0000 7f46801aa740 1 -- 172.21.15.94:0/1154602782 wait complete. 2024-06-03T01:14:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.009+0000 7f46801aa740 20 client.15483 trim_cache size 0 max 16384 2024-06-03T01:14:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:14:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:14:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.009+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f465c0a2da0 msgr2=0x7f465c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.009+0000 7f467d200640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f465c0a2da0 0x7f465c0a5260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x564908e23d10 tx=0x7f46680d36e0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.009+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4678102c50 msgr2=0x7f4678198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4678102c50 0x7f4678198210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f466800e9f0 tx=0x7f466800edf0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 shutdown_connections 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f465c0a2da0 0x7f465c0a5260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f467810c020 0x7f467819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4678103570 0x7f4678198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 --2- 172.21.15.94:0/1871765521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4678102c50 0x7f4678198210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 >> 172.21.15.94:0/1871765521 conn(0x7f4678076c20 msgr2=0x7f46780fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 shutdown_connections 2024-06-03T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:09.010+0000 7f467d200640 1 -- 172.21.15.94:0/1871765521 wait complete. 2024-06-03T01:14:09.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:08 smithi107 ceph-mon[36622]: pgmap v116: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-03T01:14:09.070 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82189 2.50 MB/sec execute 16 sec latency 614.274 ms 2024-06-03T01:14:10.071 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82189 2.35 MB/sec execute 17 sec latency 1614.422 ms 2024-06-03T01:14:10.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:10 smithi094 ceph-mon[32119]: pgmap v117: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-06-03T01:14:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:10 smithi179 ceph-mon[37041]: pgmap v117: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-06-03T01:14:11.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:10 smithi107 ceph-mon[36622]: pgmap v117: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-06-03T01:14:11.071 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82189 2.22 MB/sec execute 18 sec latency 2614.564 ms 2024-06-03T01:14:12.071 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82189 2.10 MB/sec execute 19 sec latency 3614.711 ms 2024-06-03T01:14:12.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:12 smithi094 ceph-mon[32119]: pgmap v118: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-06-03T01:14:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:12 smithi179 ceph-mon[37041]: pgmap v118: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-06-03T01:14:13.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:12 smithi107 ceph-mon[36622]: pgmap v118: 129 pgs: 129 active+clean; 42 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-06-03T01:14:13.071 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82189 2.00 MB/sec execute 20 sec latency 4614.857 ms 2024-06-03T01:14:14.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:13 smithi107 ceph-mon[36622]: pgmap v119: 129 pgs: 129 active+clean; 40 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-03T01:14:14.071 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82189 1.90 MB/sec execute 21 sec latency 5615.006 ms 2024-06-03T01:14:14.102 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:13 smithi094 ceph-mon[32119]: pgmap v119: 129 pgs: 129 active+clean; 40 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-03T01:14:14.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:13 smithi179 ceph-mon[37041]: pgmap v119: 129 pgs: 129 active+clean; 40 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-03T01:14:15.071 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82189 1.82 MB/sec execute 22 sec latency 6615.149 ms 2024-06-03T01:14:16.071 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82209 1.74 MB/sec execute 23 sec latency 7589.858 ms 2024-06-03T01:14:16.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:15 smithi107 ceph-mon[36622]: pgmap v120: 129 pgs: 129 active+clean; 40 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 888 KiB/s wr, 108 op/s 2024-06-03T01:14:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:15 smithi094 ceph-mon[32119]: pgmap v120: 129 pgs: 129 active+clean; 40 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 888 KiB/s wr, 108 op/s 2024-06-03T01:14:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:15 smithi179 ceph-mon[37041]: pgmap v120: 129 pgs: 129 active+clean; 40 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 888 KiB/s wr, 108 op/s 2024-06-03T01:14:17.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82302 1.67 MB/sec execute 24 sec latency 922.406 ms 2024-06-03T01:14:18.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82302 1.60 MB/sec execute 25 sec latency 1922.488 ms 2024-06-03T01:14:18.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:17 smithi107 ceph-mon[36622]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-06-03T01:14:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:17 smithi094 ceph-mon[32119]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-06-03T01:14:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:17 smithi179 ceph-mon[37041]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-06-03T01:14:19.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82302 1.54 MB/sec execute 26 sec latency 2922.576 ms 2024-06-03T01:14:20.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82302 1.48 MB/sec execute 27 sec latency 3922.720 ms 2024-06-03T01:14:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:19 smithi107 ceph-mon[36622]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 677 KiB/s wr, 76 op/s 2024-06-03T01:14:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:19 smithi094 ceph-mon[32119]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 677 KiB/s wr, 76 op/s 2024-06-03T01:14:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:19 smithi179 ceph-mon[37041]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 677 KiB/s wr, 76 op/s 2024-06-03T01:14:21.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82317 1.43 MB/sec execute 28 sec latency 4901.254 ms 2024-06-03T01:14:22.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82358 1.39 MB/sec execute 29 sec latency 953.919 ms 2024-06-03T01:14:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 -- 172.21.15.94:0/598400651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c108860 msgr2=0x7f648c114ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 --2- 172.21.15.94:0/598400651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c108860 0x7f648c114ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6484009c10 tx=0x7f648402a6e0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 -- 172.21.15.94:0/598400651 shutdown_connections 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 --2- 172.21.15.94:0/598400651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c108860 0x7f648c114ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 --2- 172.21.15.94:0/598400651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648c107ee0 0x7f648c108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 --2- 172.21.15.94:0/598400651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 0x7f648c10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 -- 172.21.15.94:0/598400651 >> 172.21.15.94:0/598400651 conn(0x7f648c077780 msgr2=0x7f648c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 -- 172.21.15.94:0/598400651 shutdown_connections 2024-06-03T01:14:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.150+0000 7f6491e00640 1 -- 172.21.15.94:0/598400651 wait complete. 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.151+0000 7f6491e00640 1 Processor -- start 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.151+0000 7f6491e00640 1 -- start start 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c107ee0 0x7f648c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648c108860 0x7f648c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 0x7f648c1a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1181a0 con 0x7f648c107ee0 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f648c118020 con 0x7f648c10de70 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f648c118320 con 0x7f648c108860 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6490a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c107ee0 0x7f648c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 0x7f648c1a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6483e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648c108860 0x7f648c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 0x7f648c1a2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48564/0 (socket says 172.21.15.94:48564) 2024-06-03T01:14:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.152+0000 7f6491400640 1 -- 172.21.15.94:0/2505986083 learned_addr learned my addr 172.21.15.94:0/2505986083 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491400640 1 -- 172.21.15.94:0/2505986083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648c108860 msgr2=0x7f648c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491400640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648c108860 0x7f648c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491400640 1 -- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c107ee0 msgr2=0x7f648c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491400640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c107ee0 0x7f648c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491400640 1 -- 172.21.15.94:0/2505986083 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c1a3670 con 0x7f648c10de70 2024-06-03T01:14:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491400640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 0x7f648c1a2f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f648402ac00 tx=0x7f64840305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f648a000640 1 -- 172.21.15.94:0/2505986083 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484041020 con 0x7f648c10de70 2024-06-03T01:14:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c1a3900 con 0x7f648c10de70 2024-06-03T01:14:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f648a000640 1 -- 172.21.15.94:0/2505986083 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6484003e50 con 0x7f648c10de70 2024-06-03T01:14:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f648a000640 1 -- 172.21.15.94:0/2505986083 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484036040 con 0x7f648c10de70 2024-06-03T01:14:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.153+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1ab1f0 con 0x7f648c10de70 2024-06-03T01:14:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.156+0000 7f649620c740 1 Processor -- start 2024-06-03T01:14:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f648a000640 1 -- 172.21.15.94:0/2505986083 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6484044020 con 0x7f648c10de70 2024-06-03T01:14:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f649620c740 1 -- start start 2024-06-03T01:14:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f649620c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f649620c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f649620c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f649620c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56102440e420 con 0x5610244b4f80 2024-06-03T01:14:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f649620c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5610243b5c50 con 0x5610244aa200 2024-06-03T01:14:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.157+0000 7f649620c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56102440a3e0 con 0x5610244ab770 2024-06-03T01:14:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6483e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6490a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6483e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:33768/0 (socket says 172.21.15.94:33768) 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6483e00640 1 -- 172.21.15.94:0/847456452 learned_addr learned my addr 172.21.15.94:0/847456452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6483e00640 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6490a00640 1 -- 172.21.15.94:0/847456452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 msgr2=0x5610244a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6490a00640 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6490a00640 1 -- 172.21.15.94:0/847456452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 msgr2=0x5610244b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6490a00640 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6490a00640 1 -- 172.21.15.94:0/847456452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561024122210 con 0x5610244aa200 2024-06-03T01:14:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f6490a00640 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244aa680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f647c009c60 tx=0x7f647c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ae076ef41d4e7e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.158+0000 7f648a000640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f64700a2cf0 0x7f64700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f6481c00640 1 -- 172.21.15.94:0/847456452 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c01e070 con 0x5610244aa200 2024-06-03T01:14:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f648a000640 1 -- 172.21.15.94:0/2505986083 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f64840f0480 con 0x7f648c10de70 2024-06-03T01:14:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f6481c00640 1 -- 172.21.15.94:0/847456452 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c009e10 con 0x5610244aa200 2024-06-03T01:14:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f6481c00640 1 -- 172.21.15.94:0/847456452 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c019630 con 0x5610244aa200 2024-06-03T01:14:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f6483e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f64700a2cf0 0x7f64700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f649620c740 1 -- 172.21.15.94:0/847456452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 msgr2=0x5610244aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f649620c740 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244aa680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f647c009c60 tx=0x7f647c011810 comp rx=0 tx=0).stop 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f6483e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f64700a2cf0 0x7f64700a51b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5610244a7980 tx=0x7f6474005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f649620c740 1 -- 172.21.15.94:0/847456452 shutdown_connections 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.159+0000 7f649620c740 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.160+0000 7f649620c740 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244aa680 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.160+0000 7f649620c740 1 --2- 172.21.15.94:0/847456452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.160+0000 7f649620c740 1 -- 172.21.15.94:0/847456452 >> 172.21.15.94:0/847456452 conn(0x561024472f80 msgr2=0x5610244a67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.160+0000 7f649620c740 1 -- 172.21.15.94:0/847456452 shutdown_connections 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.160+0000 7f649620c740 1 -- 172.21.15.94:0/847456452 wait complete. 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.160+0000 7f649620c740 1 -- start start 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.160+0000 7f649620c740 1 Processor -- start 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f649620c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244aa200 0x5610244b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f649620c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244ab770 0x5610244b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f649620c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f649620c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5610244b7d50 con 0x5610244b4f80 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f649620c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5610243c90b0 con 0x5610244ab770 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f649620c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5610244b7ed0 con 0x5610244aa200 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f649620c740 10 client.0 _ll_register_callbacks cb 0x5610243b4f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f6491400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f6483e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244ab770 0x5610244b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f6490a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244aa200 0x5610244b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f6491400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38292/0 (socket says 172.21.15.94:38292) 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f6483e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244ab770 0x5610244b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48584/0 (socket says 172.21.15.94:48584) 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.161+0000 7f6491400640 1 -- 172.21.15.94:0/1415775447 learned_addr learned my addr 172.21.15.94:0/1415775447 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6491400640 1 -- 172.21.15.94:0/1415775447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244aa200 msgr2=0x5610244b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6491400640 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244aa200 0x5610244b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6491400640 1 -- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244ab770 msgr2=0x5610244b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6491400640 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244ab770 0x5610244b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6491400640 1 -- 172.21.15.94:0/1415775447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610241ee620 con 0x5610244b4f80 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6490a00640 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244aa200 0x5610244b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6483e00640 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244ab770 0x5610244b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f6491400640 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b2a20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6484004af0 tx=0x7f64840bc1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22f799bacf66d00 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f647a000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f647a000640 1 -- 172.21.15.94:0/1415775447 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64840c0990 con 0x5610244b4f80 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f647a000640 1 -- 172.21.15.94:0/1415775447 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f6484031560 con 0x5610244b4f80 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f647c009e10 con 0x5610244b4f80 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f647a000640 1 -- 172.21.15.94:0/1415775447 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f64840c5250 con 0x5610244b4f80 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.162+0000 7f649620c740 10 client.15495 fetch_fsmap learned FSMap version 33 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610244c40f0 con 0x5610244b4f80 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f647a000640 1 -- 172.21.15.94:0/1415775447 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c009e10 con 0x5610244b4f80 2024-06-03T01:14:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f647a000640 1 -- 172.21.15.94:0/1415775447 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f6484004b40 con 0x5610244b4f80 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f649620c740 10 client.15495 fetch_fsmap finished waiting for FSMap version 33 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f649620c740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f649620c740 20 client.15495 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f649620c740 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5610244c5650 0x5610244c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f649620c740 4 client.15495 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56102441bb50 con 0x5610244c5650 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.163+0000 7f6490a00640 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5610244c5650 0x5610244c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.164+0000 7f6490a00640 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5610244c5650 0x5610244c7ab0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250c89619fed43fc server_cookie=808f50d57fcceda1 in_seq=0 out_seq=0 2024-06-03T01:14:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.164+0000 7f647a000640 10 client.15495 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f647a000640 1 -- 172.21.15.94:0/1415775447 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56102441bb50 con 0x5610244c5650 2024-06-03T01:14:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f647a000640 10 client.15495 handle_command_reply: tid=0 2024-06-03T01:14:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 client.15495 shutdown 2024-06-03T01:14:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 msgr2=0x5610244b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b2a20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6484004af0 tx=0x7f64840bc1e0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 shutdown_connections 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244b2a20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5610244c5650 0x5610244c7ab0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244ab770 0x5610244b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 --2- 172.21.15.94:0/1415775447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244aa200 0x5610244b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f647a000640 0 client.15495 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 >> 172.21.15.94:0/1415775447 conn(0x561024472f80 msgr2=0x5610244ac660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 shutdown_connections 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.165+0000 7f649620c740 1 -- 172.21.15.94:0/1415775447 wait complete. 2024-06-03T01:14:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.166+0000 7f649620c740 20 client.15495 trim_cache size 0 max 16384 2024-06-03T01:14:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 Processor -- start 2024-06-03T01:14:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 -- start start 2024-06-03T01:14:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1b50b0 con 0x5610244b4f80 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1b6ce0 con 0x5610244aa200 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.182+0000 7f649620c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1b5780 con 0x5610244ab770 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48588/0 (socket says 172.21.15.94:48588) 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 -- 172.21.15.94:0/4061713889 learned_addr learned my addr 172.21.15.94:0/4061713889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6491400640 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6483e00640 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 -- 172.21.15.94:0/4061713889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 msgr2=0x5610244a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 -- 172.21.15.94:0/4061713889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 msgr2=0x5610244a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 -- 172.21.15.94:0/4061713889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610244a1fd0 con 0x5610244aa200 2024-06-03T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6491400640 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.183+0000 7f6490a00640 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244b2670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f647c0099f0 tx=0x7f647c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4625fda6b7c8d42e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f6481200640 1 -- 172.21.15.94:0/4061713889 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c01e040 con 0x5610244aa200 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f6481200640 1 -- 172.21.15.94:0/4061713889 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c029070 con 0x5610244aa200 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f6481200640 1 -- 172.21.15.94:0/4061713889 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c02a7d0 con 0x5610244aa200 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 -- 172.21.15.94:0/4061713889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 msgr2=0x5610244b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244b2670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f647c0099f0 tx=0x7f647c000bc0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 -- 172.21.15.94:0/4061713889 shutdown_connections 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244b4f80 0x5610244a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244ab770 0x5610244a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 --2- 172.21.15.94:0/4061713889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244b2670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 -- 172.21.15.94:0/4061713889 >> 172.21.15.94:0/4061713889 conn(0x561024472f80 msgr2=0x5610244b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 -- 172.21.15.94:0/4061713889 shutdown_connections 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.184+0000 7f649620c740 1 -- 172.21.15.94:0/4061713889 wait complete. 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 -- start start 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 Processor -- start 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244ab770 0x5610244cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244b4f80 0x5610244ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1b6550 con 0x5610244ab770 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1b5e50 con 0x5610244aa200 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f648c1b3090 con 0x5610244b4f80 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.185+0000 7f649620c740 10 client.0 _ll_register_callbacks cb 0x7f64840308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244ab770 0x5610244cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6490a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244ab770 0x5610244cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38316/0 (socket says 172.21.15.94:38316) 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 -- 172.21.15.94:0/3162490019 learned_addr learned my addr 172.21.15.94:0/3162490019 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6490a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244cdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48590/0 (socket says 172.21.15.94:48590) 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6491400640 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244b4f80 0x5610244ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 -- 172.21.15.94:0/3162490019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244b4f80 msgr2=0x5610244ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244b4f80 0x5610244ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 -- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 msgr2=0x5610244cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6483e00640 1 -- 172.21.15.94:0/3162490019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610244bc190 con 0x5610244ab770 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.186+0000 7f6490a00640 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f6483e00640 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244ab770 0x5610244cc3b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6474009a70 tx=0x7f6474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cc33a3b974e0d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f647a000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f647a000640 1 -- 172.21.15.94:0/3162490019 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64740038c0 con 0x5610244ab770 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f647c029070 con 0x5610244ab770 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f647a000640 1 -- 172.21.15.94:0/3162490019 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f647401a050 con 0x5610244ab770 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f647a000640 1 -- 172.21.15.94:0/3162490019 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f647403a050 con 0x5610244ab770 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f649620c740 10 client.15501 fetch_fsmap learned FSMap version 33 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610244bd520 con 0x5610244ab770 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.187+0000 7f647a000640 1 -- 172.21.15.94:0/3162490019 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f647c029070 con 0x5610244ab770 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f647a000640 1 -- 172.21.15.94:0/3162490019 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f64740047a0 con 0x5610244ab770 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f649620c740 10 client.15501 fetch_fsmap finished waiting for FSMap version 33 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f649620c740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f649620c740 20 client.15501 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f649620c740 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561024530390 0x561024532780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f649620c740 4 client.15501 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561024415ea0 con 0x561024530390 2024-06-03T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.188+0000 7f6490a00640 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561024530390 0x561024532780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.189+0000 7f6490a00640 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561024530390 0x561024532780 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f62587465ea7f0 server_cookie=9d993f4d79b3621a in_seq=0 out_seq=0 2024-06-03T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.189+0000 7f647a000640 10 client.15501 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.212+0000 7f647a000640 1 -- 172.21.15.94:0/3162490019 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561024415ea0 con 0x561024530390 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.212+0000 7f647a000640 10 client.15501 handle_command_reply: tid=0 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.212+0000 7f649620c740 1 client.15501 shutdown 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.212+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244ab770 msgr2=0x5610244cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.212+0000 7f649620c740 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244ab770 0x5610244cc3b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6474009a70 tx=0x7f6474004730 comp rx=0 tx=0).stop 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 shutdown_connections 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610244b4f80 0x5610244ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610244ab770 0x5610244cc3b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561024530390 0x561024532780 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 --2- 172.21.15.94:0/3162490019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610244aa200 0x5610244cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 >> 172.21.15.94:0/3162490019 conn(0x561024472f80 msgr2=0x5610244ac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f647a000640 0 client.15501 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 shutdown_connections 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 1 -- 172.21.15.94:0/3162490019 wait complete. 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.213+0000 7f649620c740 20 client.15501 trim_cache size 0 max 16384 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f64700a2cf0 msgr2=0x7f64700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f64700a2cf0 0x7f64700a51b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x5610244a7980 tx=0x7f6474005f90 comp rx=0 tx=0).stop 2024-06-03T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 msgr2=0x7f648c1a2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 0x7f648c1a2f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f648402ac00 tx=0x7f64840305d0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 shutdown_connections 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f64700a2cf0 0x7f64700a51b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f648c10de70 0x7f648c1a2f40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f648c108860 0x7f648c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 --2- 172.21.15.94:0/2505986083 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f648c107ee0 0x7f648c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 >> 172.21.15.94:0/2505986083 conn(0x7f648c077780 msgr2=0x7f648c078890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 shutdown_connections 2024-06-03T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:22.214+0000 7f6491e00640 1 -- 172.21.15.94:0/2505986083 wait complete. 2024-06-03T01:14:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:21 smithi107 ceph-mon[36622]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 677 KiB/s wr, 76 op/s 2024-06-03T01:14:22.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:21 smithi094 ceph-mon[32119]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 677 KiB/s wr, 76 op/s 2024-06-03T01:14:22.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:21 smithi179 ceph-mon[37041]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 677 KiB/s wr, 76 op/s 2024-06-03T01:14:23.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82358 1.34 MB/sec execute 30 sec latency 1954.064 ms 2024-06-03T01:14:24.072 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82358 1.30 MB/sec execute 31 sec latency 2954.212 ms 2024-06-03T01:14:24.157 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:23 smithi179 ceph-mon[37041]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 958 KiB/s wr, 99 op/s 2024-06-03T01:14:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:23 smithi107 ceph-mon[36622]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 958 KiB/s wr, 99 op/s 2024-06-03T01:14:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:23 smithi094 ceph-mon[32119]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 958 KiB/s wr, 99 op/s 2024-06-03T01:14:25.073 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82358 1.26 MB/sec execute 32 sec latency 3954.361 ms 2024-06-03T01:14:26.073 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82375 1.22 MB/sec execute 33 sec latency 4927.346 ms 2024-06-03T01:14:26.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:25 smithi107 ceph-mon[36622]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 552 KiB/s wr, 42 op/s 2024-06-03T01:14:26.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:25 smithi094 ceph-mon[32119]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 552 KiB/s wr, 42 op/s 2024-06-03T01:14:26.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:25 smithi179 ceph-mon[37041]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 552 KiB/s wr, 42 op/s 2024-06-03T01:14:27.073 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82426 1.19 MB/sec execute 34 sec latency 943.651 ms 2024-06-03T01:14:28.073 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82426 1.15 MB/sec execute 35 sec latency 1943.826 ms 2024-06-03T01:14:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:27 smithi107 ceph-mon[36622]: pgmap v126: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 829 KiB/s wr, 67 op/s 2024-06-03T01:14:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:27 smithi094 ceph-mon[32119]: pgmap v126: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 829 KiB/s wr, 67 op/s 2024-06-03T01:14:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:27 smithi179 ceph-mon[37041]: pgmap v126: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 829 KiB/s wr, 67 op/s 2024-06-03T01:14:29.073 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82426 1.12 MB/sec execute 36 sec latency 2943.988 ms 2024-06-03T01:14:30.073 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82426 1.09 MB/sec execute 37 sec latency 3944.152 ms 2024-06-03T01:14:30.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:29 smithi107 ceph-mon[36622]: pgmap v127: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 558 KiB/s wr, 47 op/s 2024-06-03T01:14:30.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:29 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:30.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:29 smithi094 ceph-mon[32119]: pgmap v127: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 558 KiB/s wr, 47 op/s 2024-06-03T01:14:30.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:29 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:30.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:29 smithi179 ceph-mon[37041]: pgmap v127: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 558 KiB/s wr, 47 op/s 2024-06-03T01:14:30.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:29 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:31.073 INFO:tasks.workunit.client.0.smithi094.stdout: 1 82441 1.06 MB/sec execute 38 sec latency 4918.707 ms 2024-06-03T01:14:32.074 INFO:tasks.workunit.client.0.smithi094.stdout: 1 83105 1.15 MB/sec execute 39 sec latency 494.215 ms 2024-06-03T01:14:32.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:31 smithi107 ceph-mon[36622]: pgmap v128: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 558 KiB/s wr, 47 op/s 2024-06-03T01:14:32.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:31 smithi094 ceph-mon[32119]: pgmap v128: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 558 KiB/s wr, 47 op/s 2024-06-03T01:14:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:31 smithi179 ceph-mon[37041]: pgmap v128: 129 pgs: 129 active+clean; 40 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 558 KiB/s wr, 47 op/s 2024-06-03T01:14:33.074 INFO:tasks.workunit.client.0.smithi094.stdout: 1 84998 1.36 MB/sec execute 40 sec latency 13.554 ms 2024-06-03T01:14:34.074 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85770 1.44 MB/sec execute 41 sec latency 23.254 ms 2024-06-03T01:14:34.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:33 smithi107 ceph-mon[36622]: pgmap v129: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 642 KiB/s wr, 75 op/s 2024-06-03T01:14:34.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:33 smithi094 ceph-mon[32119]: pgmap v129: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 642 KiB/s wr, 75 op/s 2024-06-03T01:14:34.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:33 smithi179 ceph-mon[37041]: pgmap v129: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 642 KiB/s wr, 75 op/s 2024-06-03T01:14:35.074 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85899 1.41 MB/sec execute 42 sec latency 868.338 ms 2024-06-03T01:14:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:34 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:34 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:35.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:34 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:35.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:34 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:35.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:34 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:35.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:34 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:14:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3627468115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 msgr2=0x7f9ec0100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3627468115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 0x7f9ec0100b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009ca0 tx=0x7f9eb002a8c0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3627468115 shutdown_connections 2024-06-03T01:14:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3627468115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 0x7f9ec0110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3627468115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec01010d0 0x7f9ec010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3627468115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 0x7f9ec0100b90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3627468115 >> 172.21.15.94:0/3627468115 conn(0x7f9ec00779b0 msgr2=0x7f9ec0077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3627468115 shutdown_connections 2024-06-03T01:14:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.366+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3627468115 wait complete. 2024-06-03T01:14:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 Processor -- start 2024-06-03T01:14:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 -- start start 2024-06-03T01:14:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 0x7f9ec019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec01010d0 0x7f9ec019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 0x7f9ec019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0113fc0 con 0x7f9ec01010d0 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0113e40 con 0x7f9ec010e360 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.367+0000 7f9ec5a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec0114140 con 0x7f9ec01007b0 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 0x7f9ec019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 0x7f9ec019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9eb6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec01010d0 0x7f9ec019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 0x7f9ec019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37334/0 (socket says 172.21.15.94:37334) 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 -- 172.21.15.94:0/3650525281 learned_addr learned my addr 172.21.15.94:0/3650525281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 -- 172.21.15.94:0/3650525281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 msgr2=0x7f9ec019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 0x7f9ec019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 -- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec01010d0 msgr2=0x7f9ec019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec01010d0 0x7f9ec019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 -- 172.21.15.94:0/3650525281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec019f570 con 0x7f9ec010e360 2024-06-03T01:14:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebea00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 0x7f9ec019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9eb6000640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec01010d0 0x7f9ec019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.368+0000 7f9ebf400640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 0x7f9ec019ee40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800e940 tx=0x7f9eb800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.369+0000 7f9eb7e00640 1 -- 172.21.15.94:0/3650525281 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb800ccc0 con 0x7f9ec010e360 2024-06-03T01:14:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.369+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec019f860 con 0x7f9ec010e360 2024-06-03T01:14:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.369+0000 7f9eb7e00640 1 -- 172.21.15.94:0/3650525281 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb8005500 con 0x7f9ec010e360 2024-06-03T01:14:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.369+0000 7f9eb7e00640 1 -- 172.21.15.94:0/3650525281 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8010690 con 0x7f9ec010e360 2024-06-03T01:14:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.369+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec01a70f0 con 0x7f9ec010e360 2024-06-03T01:14:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.372+0000 7f9ec884e740 1 Processor -- start 2024-06-03T01:14:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.373+0000 7f9eb7e00640 1 -- 172.21.15.94:0/3650525281 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9eb80046c0 con 0x7f9ec010e360 2024-06-03T01:14:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ec884e740 1 -- start start 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b905b0 0x559d56b8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b8f040 0x559d56b8f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b99dc0 0x559d56b9c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ec884e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559d56af3420 con 0x559d56b8f040 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ec884e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559d56a9ac50 con 0x559d56b905b0 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ec884e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559d56aef3e0 con 0x559d56b99dc0 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b8f040 0x559d56b8f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9eb6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b905b0 0x559d56b8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b8f040 0x559d56b8f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56086/0 (socket says 172.21.15.94:56086) 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9eb6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b905b0 0x559d56b8eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37350/0 (socket says 172.21.15.94:37350) 2024-06-03T01:14:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9ebea00640 1 -- 172.21.15.94:0/741598033 learned_addr learned my addr 172.21.15.94:0/741598033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.374+0000 7f9eb6000640 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b99dc0 0x559d56b9c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9ebea00640 1 -- 172.21.15.94:0/741598033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b99dc0 msgr2=0x559d56b9c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9ebea00640 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b99dc0 0x559d56b9c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9ebea00640 1 -- 172.21.15.94:0/741598033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b905b0 msgr2=0x559d56b8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9ebea00640 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b905b0 0x559d56b8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9ebea00640 1 -- 172.21.15.94:0/741598033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d56807210 con 0x559d56b8f040 2024-06-03T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9eb7e00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f9ea40a2da0 0x7f9ea40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9ebea00640 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b8f040 0x559d56b8f4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009bf0 tx=0x7f9eb0005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb07b0f77fef053 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9eb7e00640 1 -- 172.21.15.94:0/3650525281 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f9eb80cfad0 con 0x7f9ec010e360 2024-06-03T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.376+0000 7f9eb6000640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f9ea40a2da0 0x7f9ea40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.375+0000 7f9eafe00640 1 -- 172.21.15.94:0/741598033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0049070 con 0x559d56b8f040 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.376+0000 7f9eafe00640 1 -- 172.21.15.94:0/741598033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb0036040 con 0x559d56b8f040 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.376+0000 7f9eafe00640 1 -- 172.21.15.94:0/741598033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb002fd80 con 0x559d56b8f040 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.376+0000 7f9ec884e740 1 -- 172.21.15.94:0/741598033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b8f040 msgr2=0x559d56b8f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.376+0000 7f9eb6000640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f9ea40a2da0 0x7f9ea40a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x559d56b8c7c0 tx=0x7f9ea8005850 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.376+0000 7f9ec884e740 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b8f040 0x559d56b8f4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009bf0 tx=0x7f9eb0005f60 comp rx=0 tx=0).stop 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 -- 172.21.15.94:0/741598033 shutdown_connections 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b99dc0 0x559d56b9c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b8f040 0x559d56b8f4c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 --2- 172.21.15.94:0/741598033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b905b0 0x559d56b8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 -- 172.21.15.94:0/741598033 >> 172.21.15.94:0/741598033 conn(0x559d56b57f80 msgr2=0x559d56b8b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 -- 172.21.15.94:0/741598033 shutdown_connections 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 -- 172.21.15.94:0/741598033 wait complete. 2024-06-03T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 -- start start 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.377+0000 7f9ec884e740 1 Processor -- start 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b96db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56b97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ec884e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559d56b9c9b0 con 0x559d56b905b0 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ec884e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559d56aae0b0 con 0x559d56b99dc0 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ec884e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559d56b9cb30 con 0x559d56b8f040 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ec884e740 10 client.0 _ll_register_callbacks cb 0x559d56a99f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ebf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56b97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9eb6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ebf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56b97830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37366/0 (socket says 172.21.15.94:37366) 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ebf400640 1 -- 172.21.15.94:0/757008052 learned_addr learned my addr 172.21.15.94:0/757008052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.378+0000 7f9ebea00640 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b96db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ebf400640 1 -- 172.21.15.94:0/757008052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 msgr2=0x559d56b96db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ebf400640 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b96db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ebf400640 1 -- 172.21.15.94:0/757008052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 msgr2=0x559d56b972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ebf400640 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ebf400640 1 -- 172.21.15.94:0/757008052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d568d3620 con 0x559d56b99dc0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ebf400640 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56b97830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8098090 tx=0x7f9eb809b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10e26a06bdf154b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9eacc00640 1 -- 172.21.15.94:0/757008052 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8086040 con 0x559d56b99dc0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9eacc00640 1 -- 172.21.15.94:0/757008052 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f9eb80a4610 con 0x559d56b99dc0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb0036040 con 0x559d56b99dc0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9eacc00640 1 -- 172.21.15.94:0/757008052 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9eb809ffe0 con 0x559d56b99dc0 2024-06-03T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ec884e740 10 client.25637 fetch_fsmap learned FSMap version 35 2024-06-03T01:14:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.379+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559d56ba8cf0 con 0x559d56b99dc0 2024-06-03T01:14:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9eacc00640 1 -- 172.21.15.94:0/757008052 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb0036040 con 0x559d56b99dc0 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9eacc00640 1 -- 172.21.15.94:0/757008052 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f9eb809ac80 con 0x559d56b99dc0 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9ec884e740 10 client.25637 fetch_fsmap finished waiting for FSMap version 35 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9ec884e740 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9ec884e740 20 client.25637 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9ec884e740 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56baa220 0x559d56bac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9ec884e740 4 client.25637 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d56b00b50 con 0x559d56baa220 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.380+0000 7f9ebea00640 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56baa220 0x559d56bac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.381+0000 7f9ebea00640 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56baa220 0x559d56bac650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd518be7742f05b server_cookie=e76d2d09439e5f1f in_seq=0 out_seq=0 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.381+0000 7f9eacc00640 10 client.25637 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9eacc00640 1 -- 172.21.15.94:0/757008052 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d56b00b50 con 0x559d56baa220 2024-06-03T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9eacc00640 10 client.25637 handle_command_reply: tid=0 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 client.25637 shutdown 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 msgr2=0x559d56b97830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56b97830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8098090 tx=0x7f9eb809b610 comp rx=0 tx=0).stop 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 shutdown_connections 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56baa220 0x559d56bac650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56b97830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9eacc00640 0 client.25637 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 --2- 172.21.15.94:0/757008052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b96db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 >> 172.21.15.94:0/757008052 conn(0x559d56b57f80 msgr2=0x559d56b91430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 shutdown_connections 2024-06-03T01:14:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 1 -- 172.21.15.94:0/757008052 wait complete. 2024-06-03T01:14:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.382+0000 7f9ec884e740 20 client.25637 trim_cache size 0 max 16384 2024-06-03T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.399+0000 7f9ec884e740 1 Processor -- start 2024-06-03T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.399+0000 7f9ec884e740 1 -- start start 2024-06-03T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b8f040 0x559d56b8b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b905b0 0x559d56bb1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b99dc0 0x559d56bb24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ec884e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec01b0f10 con 0x559d56b99dc0 2024-06-03T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ec884e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec01b2ab0 con 0x559d56b8f040 2024-06-03T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ec884e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec01b15e0 con 0x559d56b905b0 2024-06-03T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b8f040 0x559d56b8b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b99dc0 0x559d56bb24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b8f040 0x559d56b8b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37368/0 (socket says 172.21.15.94:37368) 2024-06-03T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 -- 172.21.15.94:0/2652433021 learned_addr learned my addr 172.21.15.94:0/2652433021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9eb6000640 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b905b0 0x559d56bb1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 -- 172.21.15.94:0/2652433021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b905b0 msgr2=0x559d56bb1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b905b0 0x559d56bb1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 -- 172.21.15.94:0/2652433021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b99dc0 msgr2=0x559d56bb24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b99dc0 0x559d56bb24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.400+0000 7f9ebea00640 1 -- 172.21.15.94:0/2652433021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d56b95300 con 0x559d56b8f040 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.401+0000 7f9ebea00640 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b8f040 0x559d56b8b7e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9eb002f930 tx=0x7f9eb0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=709f3ad50923bc56 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.401+0000 7f9eaf400640 1 -- 172.21.15.94:0/2652433021 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0049040 con 0x559d56b8f040 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.401+0000 7f9eaf400640 1 -- 172.21.15.94:0/2652433021 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb0036040 con 0x559d56b8f040 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.401+0000 7f9eaf400640 1 -- 172.21.15.94:0/2652433021 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0030820 con 0x559d56b8f040 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 -- 172.21.15.94:0/2652433021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b8f040 msgr2=0x559d56b8b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b8f040 0x559d56b8b7e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9eb002f930 tx=0x7f9eb0005fd0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 -- 172.21.15.94:0/2652433021 shutdown_connections 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b99dc0 0x559d56bb24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b905b0 0x559d56bb1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 --2- 172.21.15.94:0/2652433021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b8f040 0x559d56b8b7e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 -- 172.21.15.94:0/2652433021 >> 172.21.15.94:0/2652433021 conn(0x559d56b57f80 msgr2=0x559d56b8b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 -- 172.21.15.94:0/2652433021 shutdown_connections 2024-06-03T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 -- 172.21.15.94:0/2652433021 wait complete. 2024-06-03T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.402+0000 7f9ec884e740 1 -- start start 2024-06-03T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 1 Processor -- start 2024-06-03T01:14:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b9d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b9d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56c12f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec01b23b0 con 0x559d56b905b0 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec01b1cb0 con 0x559d56b99dc0 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec01aeef0 con 0x559d56b8f040 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ec884e740 10 client.0 _ll_register_callbacks cb 0x7f9eb8004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b9d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b9d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49256/0 (socket says 172.21.15.94:49256) 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ebea00640 1 -- 172.21.15.94:0/59003551 learned_addr learned my addr 172.21.15.94:0/59003551 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9eb6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b9d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.403+0000 7f9ebf400640 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56c12f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9ebea00640 1 -- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 msgr2=0x559d56c12f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9ebea00640 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56c12f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9ebea00640 1 -- 172.21.15.94:0/59003551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 msgr2=0x559d56b9d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9ebea00640 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b9d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9ebea00640 1 -- 172.21.15.94:0/59003551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d56c13510 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9ebea00640 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b9d380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0031930 tx=0x7f9eb0005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=203758302220b238 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9eacc00640 1 -- 172.21.15.94:0/59003551 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb005e020 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb0036040 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9eacc00640 1 -- 172.21.15.94:0/59003551 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f9eb0031660 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.404+0000 7f9eacc00640 1 -- 172.21.15.94:0/59003551 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9eb0069050 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 10 client.25657 fetch_fsmap learned FSMap version 35 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559d56c14800 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9eacc00640 1 -- 172.21.15.94:0/59003551 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb0036040 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9eacc00640 1 -- 172.21.15.94:0/59003551 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f9eb0059030 con 0x559d56b8f040 2024-06-03T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 10 client.25657 fetch_fsmap finished waiting for FSMap version 35 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 10 client.25657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 20 client.25657 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56c14b90 0x559d56c16f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 4 client.25657 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.405+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d56afaea0 con 0x559d56c14b90 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.406+0000 7f9eb6000640 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56c14b90 0x559d56c16f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.406+0000 7f9eb6000640 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56c14b90 0x559d56c16f80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b395abfd0048f388 server_cookie=1a0c67af60a5239a in_seq=0 out_seq=0 2024-06-03T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.406+0000 7f9eacc00640 10 client.25657 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.434+0000 7f9eacc00640 1 -- 172.21.15.94:0/59003551 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559d56afaea0 con 0x559d56c14b90 2024-06-03T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.434+0000 7f9eacc00640 10 client.25657 handle_command_reply: tid=0 2024-06-03T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.434+0000 7f9ec884e740 1 client.25657 shutdown 2024-06-03T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 msgr2=0x559d56b9d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b9d380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0031930 tx=0x7f9eb0005f30 comp rx=0 tx=0).stop 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 shutdown_connections 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559d56c14b90 0x559d56c16f80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559d56b99dc0 0x559d56c12f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559d56b905b0 0x559d56b9d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9eacc00640 0 client.25657 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 --2- 172.21.15.94:0/59003551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559d56b8f040 0x559d56b9d380 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 >> 172.21.15.94:0/59003551 conn(0x559d56b57f80 msgr2=0x559d56b91430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 shutdown_connections 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 1 -- 172.21.15.94:0/59003551 wait complete. 2024-06-03T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.435+0000 7f9ec884e740 20 client.25657 trim_cache size 0 max 16384 2024-06-03T01:14:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:14:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:14:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f9ea40a2da0 msgr2=0x7f9ea40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f9ea40a2da0 0x7f9ea40a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x559d56b8c7c0 tx=0x7f9ea8005850 comp rx=0 tx=0).stop 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 msgr2=0x7f9ec019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 0x7f9ec019ee40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800e940 tx=0x7f9eb800ee00 comp rx=0 tx=0).stop 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 shutdown_connections 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f9ea40a2da0 0x7f9ea40a5260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ec010e360 0x7f9ec019ee40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ec01010d0 0x7f9ec019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 --2- 172.21.15.94:0/3650525281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ec01007b0 0x7f9ec019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 >> 172.21.15.94:0/3650525281 conn(0x7f9ec00779b0 msgr2=0x7f9ec010f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 shutdown_connections 2024-06-03T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:35.436+0000 7f9ec5a00640 1 -- 172.21.15.94:0/3650525281 wait complete. 2024-06-03T01:14:36.074 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85916 1.38 MB/sec execute 43 sec latency 1840.178 ms 2024-06-03T01:14:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:35 smithi107 ceph-mon[36622]: pgmap v130: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 361 KiB/s wr, 52 op/s 2024-06-03T01:14:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:35 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:35 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:14:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:35 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:14:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:35 smithi094 ceph-mon[32119]: pgmap v130: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 361 KiB/s wr, 52 op/s 2024-06-03T01:14:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:35 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:35 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:14:36.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:35 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:14:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:35 smithi179 ceph-mon[37041]: pgmap v130: 129 pgs: 129 active+clean; 39 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 361 KiB/s wr, 52 op/s 2024-06-03T01:14:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:35 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:14:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:35 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:14:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:35 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:14:37.074 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85967 1.35 MB/sec execute 44 sec latency 953.050 ms 2024-06-03T01:14:38.075 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85967 1.32 MB/sec execute 45 sec latency 1953.147 ms 2024-06-03T01:14:38.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:37 smithi107 ceph-mon[36622]: pgmap v131: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 897 KiB/s wr, 112 op/s 2024-06-03T01:14:38.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:37 smithi094 ceph-mon[32119]: pgmap v131: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 897 KiB/s wr, 112 op/s 2024-06-03T01:14:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:37 smithi179 ceph-mon[37041]: pgmap v131: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 897 KiB/s wr, 112 op/s 2024-06-03T01:14:39.075 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85967 1.29 MB/sec execute 46 sec latency 2953.295 ms 2024-06-03T01:14:40.075 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85967 1.26 MB/sec execute 47 sec latency 3953.445 ms 2024-06-03T01:14:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:39 smithi107 ceph-mon[36622]: pgmap v132: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 620 KiB/s wr, 86 op/s 2024-06-03T01:14:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:39 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:39 smithi094 ceph-mon[32119]: pgmap v132: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 620 KiB/s wr, 86 op/s 2024-06-03T01:14:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:39 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:39 smithi179 ceph-mon[37041]: pgmap v132: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 620 KiB/s wr, 86 op/s 2024-06-03T01:14:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:39 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:14:41.075 INFO:tasks.workunit.client.0.smithi094.stdout: 1 85989 1.24 MB/sec execute 48 sec latency 4924.687 ms 2024-06-03T01:14:42.075 INFO:tasks.workunit.client.0.smithi094.stdout: 1 87675 1.36 MB/sec execute 49 sec latency 13.674 ms 2024-06-03T01:14:42.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:41 smithi107 ceph-mon[36622]: pgmap v133: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 620 KiB/s wr, 86 op/s 2024-06-03T01:14:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:41 smithi094 ceph-mon[32119]: pgmap v133: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 620 KiB/s wr, 86 op/s 2024-06-03T01:14:42.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:41 smithi179 ceph-mon[37041]: pgmap v133: 129 pgs: 129 active+clean; 43 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 620 KiB/s wr, 86 op/s 2024-06-03T01:14:43.075 INFO:tasks.workunit.client.0.smithi094.stdout: 1 89074 1.56 MB/sec execute 50 sec latency 13.842 ms 2024-06-03T01:14:44.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 89984 1.57 MB/sec execute 51 sec latency 31.422 ms 2024-06-03T01:14:44.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:43 smithi107 ceph-mon[36622]: pgmap v134: 129 pgs: 129 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 958 KiB/s wr, 118 op/s 2024-06-03T01:14:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:43 smithi094 ceph-mon[32119]: pgmap v134: 129 pgs: 129 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 958 KiB/s wr, 118 op/s 2024-06-03T01:14:44.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:43 smithi179 ceph-mon[37041]: pgmap v134: 129 pgs: 129 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 958 KiB/s wr, 118 op/s 2024-06-03T01:14:45.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 91932 1.76 MB/sec execute 52 sec latency 14.951 ms 2024-06-03T01:14:46.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92835 1.84 MB/sec execute 53 sec latency 13.024 ms 2024-06-03T01:14:46.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:45 smithi107 ceph-mon[36622]: pgmap v135: 129 pgs: 129 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 874 KiB/s wr, 91 op/s 2024-06-03T01:14:46.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:45 smithi094 ceph-mon[32119]: pgmap v135: 129 pgs: 129 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 874 KiB/s wr, 91 op/s 2024-06-03T01:14:46.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:45 smithi179 ceph-mon[37041]: pgmap v135: 129 pgs: 129 active+clean; 42 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 874 KiB/s wr, 91 op/s 2024-06-03T01:14:47.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92936 1.80 MB/sec execute 54 sec latency 920.532 ms 2024-06-03T01:14:48.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92936 1.77 MB/sec execute 55 sec latency 1920.699 ms 2024-06-03T01:14:48.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:47 smithi107 ceph-mon[36622]: pgmap v136: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-03T01:14:48.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:47 smithi094 ceph-mon[32119]: pgmap v136: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-03T01:14:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:47 smithi179 ceph-mon[37041]: pgmap v136: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-03T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 -- 172.21.15.94:0/1920948838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc0767f0 msgr2=0x7f72fc112300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 --2- 172.21.15.94:0/1920948838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc0767f0 0x7f72fc112300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009c10 tx=0x7f72ec02a6e0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 -- 172.21.15.94:0/1920948838 shutdown_connections 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 --2- 172.21.15.94:0/1920948838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72fc112840 0x7f72fc114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 --2- 172.21.15.94:0/1920948838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc0767f0 0x7f72fc112300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 --2- 172.21.15.94:0/1920948838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc075ed0 0x7f72fc0762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 -- 172.21.15.94:0/1920948838 >> 172.21.15.94:0/1920948838 conn(0x7f72fc0ff6d0 msgr2=0x7f72fc101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 -- 172.21.15.94:0/1920948838 shutdown_connections 2024-06-03T01:14:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.578+0000 7f7304000640 1 -- 172.21.15.94:0/1920948838 wait complete. 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.579+0000 7f7304000640 1 Processor -- start 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7304000640 1 -- start start 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7304000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc075ed0 0x7f72fc1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7304000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72fc0767f0 0x7f72fc1a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7304000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc112840 0x7f72fc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7304000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc118340 con 0x7f72fc112840 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7304000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1181c0 con 0x7f72fc0767f0 2024-06-03T01:14:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7304000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1184c0 con 0x7f72fc075ed0 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7301600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc075ed0 0x7f72fc1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7301600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc075ed0 0x7f72fc1a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:60956/0 (socket says 172.21.15.94:60956) 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7301600640 1 -- 172.21.15.94:0/1937042994 learned_addr learned my addr 172.21.15.94:0/1937042994 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.580+0000 7f7300c00640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72fc0767f0 0x7f72fc1a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f7302000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc112840 0x7f72fc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f7301600640 1 -- 172.21.15.94:0/1937042994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72fc0767f0 msgr2=0x7f72fc1a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f7301600640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72fc0767f0 0x7f72fc1a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f7301600640 1 -- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc112840 msgr2=0x7f72fc19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f7301600640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc112840 0x7f72fc19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f7301600640 1 -- 172.21.15.94:0/1937042994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc19ef60 con 0x7f72fc075ed0 2024-06-03T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f7301600640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc075ed0 0x7f72fc1a45e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f72f000e9c0 tx=0x7f72f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f72fa000640 1 -- 172.21.15.94:0/1937042994 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f000ccc0 con 0x7f72fc075ed0 2024-06-03T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f72fa000640 1 -- 172.21.15.94:0/1937042994 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f0005500 con 0x7f72fc075ed0 2024-06-03T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.581+0000 7f72fa000640 1 -- 172.21.15.94:0/1937042994 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0010430 con 0x7f72fc075ed0 2024-06-03T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.582+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc19f250 con 0x7f72fc075ed0 2024-06-03T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.582+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc1ab560 con 0x7f72fc075ed0 2024-06-03T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 Processor -- start 2024-06-03T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 -- start start 2024-06-03T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f72fa000640 1 -- 172.21.15.94:0/1937042994 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f72f00046c0 con 0x7f72fc075ed0 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638509035b0 0x563850901b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563850902040 0x5638509024c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56385090cdc0 0x56385090f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563850866420 con 0x5638509035b0 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56385080dc50 con 0x56385090cdc0 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.585+0000 7f7306e73740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5638508623e0 con 0x563850902040 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7301600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563850902040 0x5638509024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7302000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638509035b0 0x563850901b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7301600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563850902040 0x5638509024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:60960/0 (socket says 172.21.15.94:60960) 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7301600640 1 -- 172.21.15.94:0/596840430 learned_addr learned my addr 172.21.15.94:0/596840430 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7300c00640 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56385090cdc0 0x56385090f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7302000640 1 -- 172.21.15.94:0/596840430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563850902040 msgr2=0x5638509024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7302000640 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563850902040 0x5638509024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7302000640 1 -- 172.21.15.94:0/596840430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56385090cdc0 msgr2=0x56385090f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7302000640 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56385090cdc0 0x56385090f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7302000640 1 -- 172.21.15.94:0/596840430 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56385057a210 con 0x5638509035b0 2024-06-03T01:14:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.586+0000 7f7300c00640 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56385090cdc0 0x56385090f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f7302000640 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638509035b0 0x563850901b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009b50 tx=0x7f72f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f314f0cf7b4b95f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f72ea600640 1 -- 172.21.15.94:0/596840430 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4004050 con 0x5638509035b0 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f72ea600640 1 -- 172.21.15.94:0/596840430 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f4005500 con 0x5638509035b0 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f72ea600640 1 -- 172.21.15.94:0/596840430 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4004d30 con 0x5638509035b0 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f72fa000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f72e40a2e20 0x7f72e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f7306e73740 1 -- 172.21.15.94:0/596840430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638509035b0 msgr2=0x563850901b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f7306e73740 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638509035b0 0x563850901b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009b50 tx=0x7f72f4011810 comp rx=0 tx=0).stop 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.587+0000 7f72fa000640 1 -- 172.21.15.94:0/1937042994 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f72f00cfa10 con 0x7f72fc075ed0 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7301600640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f72e40a2e20 0x7f72e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 -- 172.21.15.94:0/596840430 shutdown_connections 2024-06-03T01:14:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56385090cdc0 0x56385090f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563850902040 0x5638509024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 --2- 172.21.15.94:0/596840430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5638509035b0 0x563850901b00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 -- 172.21.15.94:0/596840430 >> 172.21.15.94:0/596840430 conn(0x5638508caf80 msgr2=0x5638508fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 -- 172.21.15.94:0/596840430 shutdown_connections 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 -- 172.21.15.94:0/596840430 wait complete. 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7301600640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f72e40a2e20 0x7f72e40a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72f0098030 tx=0x7f72f00d3550 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 -- start start 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.588+0000 7f7306e73740 1 Processor -- start 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7306e73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850909dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7306e73740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x56385090a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7306e73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x56385090a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7306e73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56385090fb90 con 0x56385090cdc0 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7306e73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5638508210b0 con 0x563850902040 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7306e73740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56385090fd10 con 0x5638509035b0 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7306e73740 10 client.0 _ll_register_callbacks cb 0x56385080cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7302000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x56385090a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7300c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x56385090a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7300c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x56385090a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:60970/0 (socket says 172.21.15.94:60970) 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7300c00640 1 -- 172.21.15.94:0/3931393305 learned_addr learned my addr 172.21.15.94:0/3931393305 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.589+0000 7f7302000640 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850909dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f7300c00640 1 -- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 msgr2=0x563850909dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f7300c00640 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850909dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f7300c00640 1 -- 172.21.15.94:0/3931393305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 msgr2=0x56385090a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f7300c00640 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x56385090a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f7300c00640 1 -- 172.21.15.94:0/3931393305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563850646620 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f7302000640 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850909dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f7300c00640 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x56385090a300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72ec034040 tx=0x7f72ec030cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2d5a0c663328dd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f72dea00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f72dea00640 1 -- 172.21.15.94:0/3931393305 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec036070 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.590+0000 7f72dea00640 1 -- 172.21.15.94:0/3931393305 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f72ec0303a0 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.591+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f4005500 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.591+0000 7f72dea00640 1 -- 172.21.15.94:0/3931393305 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f72ec0305a0 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.591+0000 7f7306e73740 10 client.25669 fetch_fsmap learned FSMap version 36 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.591+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56385091bed0 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.591+0000 7f72dea00640 1 -- 172.21.15.94:0/3931393305 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f4005500 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.591+0000 7f72dea00640 1 -- 172.21.15.94:0/3931393305 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f72ec03c040 con 0x5638509035b0 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.592+0000 7f7306e73740 10 client.25669 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.592+0000 7f7306e73740 10 client.25669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.592+0000 7f7306e73740 20 client.25669 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.592+0000 7f7306e73740 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56385091d3e0 0x56385091f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.592+0000 7f7306e73740 4 client.25669 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.592+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563850873b50 con 0x56385091d3e0 2024-06-03T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.592+0000 7f7302000640 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56385091d3e0 0x56385091f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.593+0000 7f7302000640 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56385091d3e0 0x56385091f7d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63a9218adb0f4001 server_cookie=1cacd2dc39dc284e in_seq=0 out_seq=0 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.593+0000 7f72dea00640 10 client.25669 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.593+0000 7f72dea00640 1 -- 172.21.15.94:0/3931393305 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563850873b50 con 0x56385091d3e0 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.593+0000 7f72dea00640 10 client.25669 handle_command_reply: tid=0 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.593+0000 7f7306e73740 1 client.25669 shutdown 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 msgr2=0x56385090a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x56385090a300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f72ec034040 tx=0x7f72ec030cd0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 shutdown_connections 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x56385090a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x56385090a300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56385091d3e0 0x56385091f7d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 --2- 172.21.15.94:0/3931393305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850909dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f72dea00640 0 client.25669 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 >> 172.21.15.94:0/3931393305 conn(0x5638508caf80 msgr2=0x563850904460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 shutdown_connections 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 1 -- 172.21.15.94:0/3931393305 wait complete. 2024-06-03T01:14:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.594+0000 7f7306e73740 20 client.25669 trim_cache size 0 max 16384 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.610+0000 7f7306e73740 1 Processor -- start 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7306e73740 1 -- start start 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7306e73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563850902040 0x56385090a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7306e73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638509035b0 0x5638509250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7306e73740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56385090cdc0 0x563850925620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7306e73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1b5390 con 0x563850902040 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7306e73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1b6f00 con 0x5638509035b0 2024-06-03T01:14:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7306e73740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1b5a60 con 0x56385090cdc0 2024-06-03T01:14:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7302000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563850902040 0x56385090a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7302000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563850902040 0x56385090a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37284/0 (socket says 172.21.15.94:37284) 2024-06-03T01:14:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7302000640 1 -- 172.21.15.94:0/1836268081 learned_addr learned my addr 172.21.15.94:0/1836268081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7300c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638509035b0 0x5638509250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.611+0000 7f7302000640 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56385090cdc0 0x563850925620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f7300c00640 1 -- 172.21.15.94:0/1836268081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56385090cdc0 msgr2=0x563850925620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f7300c00640 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56385090cdc0 0x563850925620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f7300c00640 1 -- 172.21.15.94:0/1836268081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563850902040 msgr2=0x56385090a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f7300c00640 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563850902040 0x56385090a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f7300c00640 1 -- 172.21.15.94:0/1836268081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563850908310 con 0x5638509035b0 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f7300c00640 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638509035b0 0x5638509250e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004820 tx=0x7f72ec02a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c6729450008751 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f72e9c00640 1 -- 172.21.15.94:0/1836268081 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec036070 con 0x5638509035b0 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f72e9c00640 1 -- 172.21.15.94:0/1836268081 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec037040 con 0x5638509035b0 2024-06-03T01:14:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.612+0000 7f72e9c00640 1 -- 172.21.15.94:0/1836268081 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec003bd0 con 0x5638509035b0 2024-06-03T01:14:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 -- 172.21.15.94:0/1836268081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638509035b0 msgr2=0x5638509250e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638509035b0 0x5638509250e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004820 tx=0x7f72ec02a6e0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 -- 172.21.15.94:0/1836268081 shutdown_connections 2024-06-03T01:14:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56385090cdc0 0x563850925620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5638509035b0 0x5638509250e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 --2- 172.21.15.94:0/1836268081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563850902040 0x56385090a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 -- 172.21.15.94:0/1836268081 >> 172.21.15.94:0/1836268081 conn(0x5638508caf80 msgr2=0x5638509050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 -- 172.21.15.94:0/1836268081 shutdown_connections 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 -- 172.21.15.94:0/1836268081 wait complete. 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.613+0000 7f7306e73740 1 -- start start 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 1 Processor -- start 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850910120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x563850910660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x5638509860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1b6830 con 0x56385090cdc0 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1b6130 con 0x563850902040 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1b3370 con 0x5638509035b0 2024-06-03T01:14:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7306e73740 10 client.0 _ll_register_callbacks cb 0x7f72f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7302000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850910120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7300c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x563850910660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7302000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x5638509860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7300c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x563850910660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:60984/0 (socket says 172.21.15.94:60984) 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7300c00640 1 -- 172.21.15.94:0/3148021965 learned_addr learned my addr 172.21.15.94:0/3148021965 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.614+0000 7f7302000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x5638509860a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37288/0 (socket says 172.21.15.94:37288) 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7302000640 1 -- 172.21.15.94:0/3148021965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 msgr2=0x563850910660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7302000640 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x563850910660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7302000640 1 -- 172.21.15.94:0/3148021965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 msgr2=0x563850910120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7302000640 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850910120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7302000640 1 -- 172.21.15.94:0/3148021965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638509865e0 con 0x56385090cdc0 2024-06-03T01:14:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7300c00640 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x563850910660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7302000640 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x5638509860a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009b20 tx=0x7f72f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1486dd7cf4400 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f72dea00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f72dea00640 1 -- 172.21.15.94:0/3148021965 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40037f0 con 0x56385090cdc0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f72dea00640 1 -- 172.21.15.94:0/3148021965 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f72f4003990 con 0x56385090cdc0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.615+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ec037040 con 0x56385090cdc0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f72dea00640 1 -- 172.21.15.94:0/3148021965 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f72f4042050 con 0x56385090cdc0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 10 client.15537 fetch_fsmap learned FSMap version 36 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563850987770 con 0x56385090cdc0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f72dea00640 1 -- 172.21.15.94:0/3148021965 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec037040 con 0x56385090cdc0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f72dea00640 1 -- 172.21.15.94:0/3148021965 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f72f4005370 con 0x56385090cdc0 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 10 client.15537 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 20 client.15537 populate_metadata read hostname 'smithi094' 2024-06-03T01:14:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563850987ca0 0x56385098a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:14:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 4 client.15537 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:14:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.616+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56385086dea0 con 0x563850987ca0 2024-06-03T01:14:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.617+0000 7f7300c00640 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563850987ca0 0x56385098a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:14:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.617+0000 7f7300c00640 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563850987ca0 0x56385098a090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8d540c48df1c37 server_cookie=3a13028303c881bc in_seq=0 out_seq=0 2024-06-03T01:14:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.617+0000 7f72dea00640 10 client.15537 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f72dea00640 1 -- 172.21.15.94:0/3148021965 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56385086dea0 con 0x563850987ca0 2024-06-03T01:14:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f72dea00640 10 client.15537 handle_command_reply: tid=0 2024-06-03T01:14:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 client.15537 shutdown 2024-06-03T01:14:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 msgr2=0x5638509860a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x5638509860a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009b20 tx=0x7f72f4005f90 comp rx=0 tx=0).stop 2024-06-03T01:14:48.657 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 shutdown_connections 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56385090cdc0 0x5638509860a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5638509035b0 0x563850910660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563850987ca0 0x56385098a090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 --2- 172.21.15.94:0/3148021965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563850902040 0x563850910120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f72dea00640 0 client.15537 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 >> 172.21.15.94:0/3148021965 conn(0x5638508caf80 msgr2=0x563850903ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 shutdown_connections 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 1 -- 172.21.15.94:0/3148021965 wait complete. 2024-06-03T01:14:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.656+0000 7f7306e73740 20 client.15537 trim_cache size 0 max 16384 2024-06-03T01:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f72e40a2e20 msgr2=0x7f72e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f72e40a2e20 0x7f72e40a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72f0098030 tx=0x7f72f00d3550 comp rx=0 tx=0).stop 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc075ed0 msgr2=0x7f72fc1a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc075ed0 0x7f72fc1a45e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f72f000e9c0 tx=0x7f72f000ee80 comp rx=0 tx=0).stop 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 shutdown_connections 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f72e40a2e20 0x7f72e40a52e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc112840 0x7f72fc19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72fc0767f0 0x7f72fc1a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 --2- 172.21.15.94:0/1937042994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72fc075ed0 0x7f72fc1a45e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 >> 172.21.15.94:0/1937042994 conn(0x7f72fc0ff6d0 msgr2=0x7f72fc101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 shutdown_connections 2024-06-03T01:14:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:14:48.657+0000 7f7304000640 1 -- 172.21.15.94:0/1937042994 wait complete. 2024-06-03T01:14:49.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92936 1.74 MB/sec execute 56 sec latency 2920.802 ms 2024-06-03T01:14:50.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92936 1.71 MB/sec execute 57 sec latency 3920.961 ms 2024-06-03T01:14:50.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:49 smithi107 ceph-mon[36622]: pgmap v137: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 723 KiB/s wr, 99 op/s 2024-06-03T01:14:50.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:49 smithi094 ceph-mon[32119]: pgmap v137: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 723 KiB/s wr, 99 op/s 2024-06-03T01:14:50.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:49 smithi179 ceph-mon[37041]: pgmap v137: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 723 KiB/s wr, 99 op/s 2024-06-03T01:14:51.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92955 1.68 MB/sec execute 58 sec latency 4892.732 ms 2024-06-03T01:14:52.076 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92992 1.65 MB/sec execute 59 sec latency 965.879 ms 2024-06-03T01:14:52.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:51 smithi107 ceph-mon[36622]: pgmap v138: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 723 KiB/s wr, 99 op/s 2024-06-03T01:14:52.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:51 smithi094 ceph-mon[32119]: pgmap v138: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 723 KiB/s wr, 99 op/s 2024-06-03T01:14:52.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:51 smithi179 ceph-mon[37041]: pgmap v138: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 723 KiB/s wr, 99 op/s 2024-06-03T01:14:53.077 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92992 1.63 MB/sec execute 60 sec latency 1966.034 ms 2024-06-03T01:14:54.077 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92992 1.60 MB/sec execute 61 sec latency 2966.174 ms 2024-06-03T01:14:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:53 smithi107 ceph-mon[36622]: pgmap v139: 129 pgs: 129 active+clean; 42 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 771 KiB/s wr, 124 op/s 2024-06-03T01:14:54.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:53 smithi094 ceph-mon[32119]: pgmap v139: 129 pgs: 129 active+clean; 42 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 771 KiB/s wr, 124 op/s 2024-06-03T01:14:54.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:53 smithi179 ceph-mon[37041]: pgmap v139: 129 pgs: 129 active+clean; 42 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 771 KiB/s wr, 124 op/s 2024-06-03T01:14:55.077 INFO:tasks.workunit.client.0.smithi094.stdout: 1 92992 1.57 MB/sec execute 62 sec latency 3966.319 ms 2024-06-03T01:14:56.077 INFO:tasks.workunit.client.0.smithi094.stdout: 1 93009 1.55 MB/sec execute 63 sec latency 4934.847 ms 2024-06-03T01:14:56.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:56 smithi107 ceph-mon[36622]: pgmap v140: 129 pgs: 129 active+clean; 42 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 92 op/s 2024-06-03T01:14:56.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:56 smithi094 ceph-mon[32119]: pgmap v140: 129 pgs: 129 active+clean; 42 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 92 op/s 2024-06-03T01:14:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:56 smithi179 ceph-mon[37041]: pgmap v140: 129 pgs: 129 active+clean; 42 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 433 KiB/s wr, 92 op/s 2024-06-03T01:14:57.077 INFO:tasks.workunit.client.0.smithi094.stdout: 1 93060 1.53 MB/sec execute 64 sec latency 940.704 ms 2024-06-03T01:14:58.077 INFO:tasks.workunit.client.0.smithi094.stdout: 1 93060 1.50 MB/sec execute 65 sec latency 1940.844 ms 2024-06-03T01:14:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:14:58 smithi107 ceph-mon[36622]: pgmap v141: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 457 KiB/s wr, 102 op/s 2024-06-03T01:14:58.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:14:58 smithi094 ceph-mon[32119]: pgmap v141: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 457 KiB/s wr, 102 op/s 2024-06-03T01:14:58.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:14:58 smithi179 ceph-mon[37041]: pgmap v141: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 457 KiB/s wr, 102 op/s 2024-06-03T01:14:59.078 INFO:tasks.workunit.client.0.smithi094.stdout: 1 93060 1.48 MB/sec execute 66 sec latency 2940.928 ms 2024-06-03T01:15:00.077 INFO:tasks.workunit.client.0.smithi094.stdout: 1 94749 1.57 MB/sec execute 67 sec latency 2977.415 ms 2024-06-03T01:15:00.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:00 smithi107 ceph-mon[36622]: pgmap v142: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-06-03T01:15:00.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:00 smithi094 ceph-mon[32119]: pgmap v142: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-06-03T01:15:00.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:00 smithi179 ceph-mon[37041]: pgmap v142: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-06-03T01:15:01.078 INFO:tasks.workunit.client.0.smithi094.stdout: 1 95523 1.64 MB/sec execute 68 sec latency 87.923 ms 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.804+0000 7f046a800640 1 -- 172.21.15.94:0/2173612150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04641077c0 msgr2=0x7f0464107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.804+0000 7f046a800640 1 --2- 172.21.15.94:0/2173612150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04641077c0 0x7f0464107ba0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0454009930 tx=0x7f045402a440 comp rx=0 tx=0).stop 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.805+0000 7f046a800640 1 -- 172.21.15.94:0/2173612150 shutdown_connections 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.805+0000 7f046a800640 1 --2- 172.21.15.94:0/2173612150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0464102170 0x7f046410e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.805+0000 7f046a800640 1 --2- 172.21.15.94:0/2173612150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 0x7f0464101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.805+0000 7f046a800640 1 --2- 172.21.15.94:0/2173612150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04641077c0 0x7f0464107ba0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.805+0000 7f046a800640 1 -- 172.21.15.94:0/2173612150 >> 172.21.15.94:0/2173612150 conn(0x7f04640fd2e0 msgr2=0x7f04640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.805+0000 7f046a800640 1 -- 172.21.15.94:0/2173612150 shutdown_connections 2024-06-03T01:15:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.805+0000 7f046a800640 1 -- 172.21.15.94:0/2173612150 wait complete. 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 Processor -- start 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 -- start start 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 0x7f0464073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0464102170 0x7f04640741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04641077c0 0x7f0464074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0464111c20 con 0x7f0464102170 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0464111aa0 con 0x7f04641077c0 2024-06-03T01:15:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f046a800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0464111da0 con 0x7f04641017b0 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f0469400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 0x7f0464073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f0468a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0464102170 0x7f04640741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f0469e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04641077c0 0x7f0464074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f0469400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 0x7f0464073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:48446/0 (socket says 172.21.15.94:48446) 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.806+0000 7f0469400640 1 -- 172.21.15.94:0/894594977 learned_addr learned my addr 172.21.15.94:0/894594977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0469400640 1 -- 172.21.15.94:0/894594977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04641077c0 msgr2=0x7f0464074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0469400640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04641077c0 0x7f0464074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0469400640 1 -- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0464102170 msgr2=0x7f04640741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0469400640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0464102170 0x7f04640741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0469400640 1 -- 172.21.15.94:0/894594977 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04641a6680 con 0x7f04641017b0 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0468a00640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0464102170 0x7f04640741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0469e00640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04641077c0 0x7f0464074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.807+0000 7f0469400640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 0x7f0464073ca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f045402a960 tx=0x7f0454003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.808+0000 7f0462000640 1 -- 172.21.15.94:0/894594977 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045403f070 con 0x7f04641017b0 2024-06-03T01:15:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.808+0000 7f0462000640 1 -- 172.21.15.94:0/894594977 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0454031c90 con 0x7f04641017b0 2024-06-03T01:15:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.808+0000 7f0462000640 1 -- 172.21.15.94:0/894594977 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454035030 con 0x7f04641017b0 2024-06-03T01:15:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.808+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04641a6910 con 0x7f04641017b0 2024-06-03T01:15:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.808+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04641a6db0 con 0x7f04641017b0 2024-06-03T01:15:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.811+0000 7f0462000640 1 -- 172.21.15.94:0/894594977 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0454042070 con 0x7f04641017b0 2024-06-03T01:15:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 Processor -- start 2024-06-03T01:15:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 -- start start 2024-06-03T01:15:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2eedcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2efb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635c2e52420 con 0x5635c2eee200 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5635c2df9c50 con 0x5635c2eef770 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f046ebcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5635c2e4e3e0 con 0x5635c2ef8f80 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.812+0000 7f0469e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2eedcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2eedcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40242/0 (socket says 172.21.15.94:40242) 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469e00640 1 -- 172.21.15.94:0/449628900 learned_addr learned my addr 172.21.15.94:0/449628900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0468a00640 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2efb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469e00640 1 -- 172.21.15.94:0/449628900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 msgr2=0x5635c2efb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469e00640 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2efb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469e00640 1 -- 172.21.15.94:0/449628900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 msgr2=0x5635c2eee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469e00640 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0469e00640 1 -- 172.21.15.94:0/449628900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c2b66210 con 0x5635c2eef770 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0468a00640 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2efb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.813+0000 7f0462000640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f044c0a2cf0 0x7f044c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.814+0000 7f0469e00640 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2eedcc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f045c0114a0 tx=0x7f045c011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823c1316e3245922 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.814+0000 7f0462000640 1 -- 172.21.15.94:0/894594977 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f04540f0170 con 0x7f04641017b0 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.814+0000 7f0452600640 1 -- 172.21.15.94:0/449628900 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c01e070 con 0x5635c2eef770 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.814+0000 7f0469400640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f044c0a2cf0 0x7f044c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.814+0000 7f0452600640 1 -- 172.21.15.94:0/449628900 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c00fa40 con 0x5635c2eef770 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.814+0000 7f0452600640 1 -- 172.21.15.94:0/449628900 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c019bf0 con 0x5635c2eef770 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 -- 172.21.15.94:0/449628900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 msgr2=0x5635c2eedcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f0469400640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f044c0a2cf0 0x7f044c0a51b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5635c2ef0ed0 tx=0x7f04540f3c20 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2eedcc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f045c0114a0 tx=0x7f045c011960 comp rx=0 tx=0).stop 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 -- 172.21.15.94:0/449628900 shutdown_connections 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2efb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 --2- 172.21.15.94:0/449628900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2eedcc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 -- 172.21.15.94:0/449628900 >> 172.21.15.94:0/449628900 conn(0x5635c2eb6f80 msgr2=0x5635c2eea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 -- 172.21.15.94:0/449628900 shutdown_connections 2024-06-03T01:15:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 -- 172.21.15.94:0/449628900 wait complete. 2024-06-03T01:15:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.815+0000 7f046ebcb740 1 -- start start 2024-06-03T01:15:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 1 Processor -- start 2024-06-03T01:15:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eee200 0x5635c2ef5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eef770 0x5635c2ef64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635c2efbd50 con 0x5635c2eef770 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5635c2e0d0b0 con 0x5635c2eee200 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5635c2efbed0 con 0x5635c2ef8f80 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f046ebcb740 10 client.0 _ll_register_callbacks cb 0x5635c2df8f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.816+0000 7f0469e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eee200 0x5635c2ef5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0468a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eef770 0x5635c2ef64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0469e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eee200 0x5635c2ef5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40246/0 (socket says 172.21.15.94:40246) 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0469e00640 1 -- 172.21.15.94:0/2355989394 learned_addr learned my addr 172.21.15.94:0/2355989394 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0469e00640 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0468a00640 1 -- 172.21.15.94:0/2355989394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 msgr2=0x5635c2ef6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0468a00640 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0468a00640 1 -- 172.21.15.94:0/2355989394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eee200 msgr2=0x5635c2ef5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0468a00640 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eee200 0x5635c2ef5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0468a00640 1 -- 172.21.15.94:0/2355989394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c2c32620 con 0x5635c2eef770 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0468a00640 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eef770 0x5635c2ef64f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5635c2eeeca0 tx=0x7f0458012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add8d0233330d2a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.817+0000 7f0446a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f0446a00640 1 -- 172.21.15.94:0/2355989394 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458003970 con 0x5635c2eef770 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f0446a00640 1 -- 172.21.15.94:0/2355989394 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f0458003b10 con 0x5635c2eef770 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045c00fa40 con 0x5635c2eef770 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f0446a00640 1 -- 172.21.15.94:0/2355989394 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0458003d10 con 0x5635c2eef770 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f046ebcb740 10 client.15549 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5635c2f08120 con 0x5635c2eef770 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f0446a00640 1 -- 172.21.15.94:0/2355989394 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c00fa40 con 0x5635c2eef770 2024-06-03T01:15:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.818+0000 7f0446a00640 1 -- 172.21.15.94:0/2355989394 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f0458010730 con 0x5635c2eef770 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f046ebcb740 10 client.15549 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f046ebcb740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f046ebcb740 20 client.15549 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f095e0 0x5635c2f0b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f046ebcb740 4 client.15549 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635c2e5fb50 con 0x5635c2f095e0 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f0469e00640 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f095e0 0x5635c2f0b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f0469e00640 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f095e0 0x5635c2f0b9d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf30c44ed89090da server_cookie=b80b695b8a3910ec in_seq=0 out_seq=0 2024-06-03T01:15:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.819+0000 7f0446a00640 10 client.15549 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f0446a00640 1 -- 172.21.15.94:0/2355989394 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5635c2e5fb50 con 0x5635c2f095e0 2024-06-03T01:15:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f0446a00640 10 client.15549 handle_command_reply: tid=0 2024-06-03T01:15:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 client.15549 shutdown 2024-06-03T01:15:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eef770 msgr2=0x5635c2ef64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eef770 0x5635c2ef64f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5635c2eeeca0 tx=0x7f0458012820 comp rx=0 tx=0).stop 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 shutdown_connections 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eef770 0x5635c2ef64f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f095e0 0x5635c2f0b9d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2355989394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eee200 0x5635c2ef5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 >> 172.21.15.94:0/2355989394 conn(0x5635c2eb6f80 msgr2=0x5635c2ef05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.820+0000 7f0446a00640 0 client.15549 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.821+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 shutdown_connections 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.821+0000 7f046ebcb740 1 -- 172.21.15.94:0/2355989394 wait complete. 2024-06-03T01:15:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.821+0000 7f046ebcb740 20 client.15549 trim_cache size 0 max 16384 2024-06-03T01:15:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.837+0000 7f046ebcb740 1 Processor -- start 2024-06-03T01:15:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f046ebcb740 1 -- start start 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2eef770 0x5635c2eea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2ef8f80 0x5635c2f11290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f046ebcb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04641b0eb0 con 0x5635c2eee200 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f046ebcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04641b2a20 con 0x5635c2ef8f80 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f046ebcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04641b1580 con 0x5635c2eef770 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0469e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2eef770 0x5635c2eea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2eef770 0x5635c2eea630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:48476/0 (socket says 172.21.15.94:48476) 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0469e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eea0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36314/0 (socket says 172.21.15.94:36314) 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 -- 172.21.15.94:0/1878264624 learned_addr learned my addr 172.21.15.94:0/1878264624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0469e00640 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2ef8f80 0x5635c2f11290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 -- 172.21.15.94:0/1878264624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2ef8f80 msgr2=0x5635c2f11290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2ef8f80 0x5635c2f11290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 -- 172.21.15.94:0/1878264624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 msgr2=0x5635c2eea0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eea0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.838+0000 7f0468a00640 1 -- 172.21.15.94:0/1878264624 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c2f0bd40 con 0x5635c2eef770 2024-06-03T01:15:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.839+0000 7f0468a00640 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2eef770 0x5635c2eea630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0458000f80 tx=0x7f0458012d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9e914bf431ad11 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.839+0000 7f0451c00640 1 -- 172.21.15.94:0/1878264624 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458030070 con 0x5635c2eef770 2024-06-03T01:15:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.839+0000 7f0451c00640 1 -- 172.21.15.94:0/1878264624 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0458010910 con 0x5635c2eef770 2024-06-03T01:15:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.839+0000 7f0451c00640 1 -- 172.21.15.94:0/1878264624 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04580053d0 con 0x5635c2eef770 2024-06-03T01:15:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 -- 172.21.15.94:0/1878264624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2eef770 msgr2=0x5635c2eea630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2eef770 0x5635c2eea630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0458000f80 tx=0x7f0458012d20 comp rx=0 tx=0).stop 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 -- 172.21.15.94:0/1878264624 shutdown_connections 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2ef8f80 0x5635c2f11290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2eef770 0x5635c2eea630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 --2- 172.21.15.94:0/1878264624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2eea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 -- 172.21.15.94:0/1878264624 >> 172.21.15.94:0/1878264624 conn(0x5635c2eb6f80 msgr2=0x5635c2ef6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 -- 172.21.15.94:0/1878264624 shutdown_connections 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 -- 172.21.15.94:0/1878264624 wait complete. 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 -- start start 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.840+0000 7f046ebcb740 1 Processor -- start 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2f01450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2f01990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f046ebcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f046ebcb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f04641b2350 con 0x5635c2eee200 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f046ebcb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04641b1c50 con 0x5635c2eef770 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f046ebcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04641aee90 con 0x5635c2ef8f80 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f046ebcb740 10 client.0 _ll_register_callbacks cb 0x7f045800f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2f01990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2f01990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40256/0 (socket says 172.21.15.94:40256) 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0469e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2f01450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 -- 172.21.15.94:0/2256947566 learned_addr learned my addr 172.21.15.94:0/2256947566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0469e00640 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 -- 172.21.15.94:0/2256947566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 msgr2=0x5635c2ef2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 -- 172.21.15.94:0/2256947566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 msgr2=0x5635c2f01450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2f01450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0468a00640 1 -- 172.21.15.94:0/2256947566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c2ef3260 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.841+0000 7f0469e00640 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2f01450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f0468a00640 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2f01990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0458010c70 tx=0x7f0458005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f6c84037cfefd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f0446a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f0446a00640 1 -- 172.21.15.94:0/2256947566 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458030040 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0458010910 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f0446a00640 1 -- 172.21.15.94:0/2256947566 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f0458010630 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f0446a00640 1 -- 172.21.15.94:0/2256947566 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f045803f050 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f046ebcb740 10 client.25679 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5635c2f73e50 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.842+0000 7f0446a00640 1 -- 172.21.15.94:0/2256947566 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0458010910 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f0446a00640 1 -- 172.21.15.94:0/2256947566 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f045802a840 con 0x5635c2eef770 2024-06-03T01:15:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f046ebcb740 10 client.25679 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f046ebcb740 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f046ebcb740 20 client.25679 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f74510 0x5635c2f76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f046ebcb740 4 client.25679 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635c2e59ea0 con 0x5635c2f74510 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.843+0000 7f0469e00640 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f74510 0x5635c2f76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.844+0000 7f0446a00640 10 client.25679 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.844+0000 7f0469e00640 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f74510 0x5635c2f76930 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9eacf9f8645a76 server_cookie=c973d32d808eb2c8 in_seq=0 out_seq=0 2024-06-03T01:15:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f0446a00640 1 -- 172.21.15.94:0/2256947566 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5635c2e59ea0 con 0x5635c2f74510 2024-06-03T01:15:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f0446a00640 10 client.25679 handle_command_reply: tid=0 2024-06-03T01:15:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 client.25679 shutdown 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 msgr2=0x5635c2f01990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2f01990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0458010c70 tx=0x7f0458005fd0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 shutdown_connections 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635c2ef8f80 0x5635c2ef2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635c2f74510 0x5635c2f76930 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635c2eef770 0x5635c2f01990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 --2- 172.21.15.94:0/2256947566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635c2eee200 0x5635c2f01450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f0446a00640 0 client.25679 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 >> 172.21.15.94:0/2256947566 conn(0x5635c2eb6f80 msgr2=0x5635c2ef5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 shutdown_connections 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 1 -- 172.21.15.94:0/2256947566 wait complete. 2024-06-03T01:15:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.874+0000 7f046ebcb740 20 client.25679 trim_cache size 0 max 16384 2024-06-03T01:15:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:15:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:15:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f044c0a2cf0 msgr2=0x7f044c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f044c0a2cf0 0x7f044c0a51b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5635c2ef0ed0 tx=0x7f04540f3c20 comp rx=0 tx=0).stop 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 msgr2=0x7f0464073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 0x7f0464073ca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f045402a960 tx=0x7f0454003b20 comp rx=0 tx=0).stop 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 shutdown_connections 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f044c0a2cf0 0x7f044c0a51b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04641077c0 0x7f0464074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0464102170 0x7f04640741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 --2- 172.21.15.94:0/894594977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04641017b0 0x7f0464073ca0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 >> 172.21.15.94:0/894594977 conn(0x7f04640fd2e0 msgr2=0x7f04640fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 shutdown_connections 2024-06-03T01:15:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:01.875+0000 7f046a800640 1 -- 172.21.15.94:0/894594977 wait complete. 2024-06-03T01:15:02.078 INFO:tasks.workunit.client.0.smithi094.stdout: 1 96078 1.68 MB/sec execute 69 sec latency 91.631 ms 2024-06-03T01:15:02.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:02 smithi107 ceph-mon[36622]: pgmap v143: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-06-03T01:15:02.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:02 smithi094 ceph-mon[32119]: pgmap v143: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-06-03T01:15:02.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:02 smithi179 ceph-mon[37041]: pgmap v143: 129 pgs: 129 active+clean; 42 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-06-03T01:15:03.078 INFO:tasks.workunit.client.0.smithi094.stdout: 1 96743 1.67 MB/sec execute 70 sec latency 51.328 ms 2024-06-03T01:15:04.078 INFO:tasks.workunit.client.0.smithi094.stdout: 1 98398 1.75 MB/sec execute 71 sec latency 13.252 ms 2024-06-03T01:15:04.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:04 smithi107 ceph-mon[36622]: pgmap v144: 129 pgs: 129 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-03T01:15:04.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:04 smithi094 ceph-mon[32119]: pgmap v144: 129 pgs: 129 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-03T01:15:04.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:04 smithi179 ceph-mon[37041]: pgmap v144: 129 pgs: 129 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-03T01:15:05.078 INFO:tasks.workunit.client.0.smithi094.stdout: 1 99689 1.88 MB/sec execute 72 sec latency 17.975 ms 2024-06-03T01:15:06.078 INFO:tasks.workunit.client.0.smithi094.stdout: 1 100103 1.86 MB/sec execute 73 sec latency 445.808 ms 2024-06-03T01:15:06.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:06 smithi094 ceph-mon[32119]: pgmap v145: 129 pgs: 129 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-03T01:15:06.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:06 smithi179 ceph-mon[37041]: pgmap v145: 129 pgs: 129 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-03T01:15:06.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:06 smithi107 ceph-mon[36622]: pgmap v145: 129 pgs: 129 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-03T01:15:07.079 INFO:tasks.workunit.client.0.smithi094.stdout: 1 101798 1.93 MB/sec execute 74 sec latency 14.611 ms 2024-06-03T01:15:08.079 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103149 2.06 MB/sec execute 75 sec latency 18.420 ms 2024-06-03T01:15:08.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:08 smithi094 ceph-mon[32119]: pgmap v146: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-06-03T01:15:08.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:08 smithi179 ceph-mon[37041]: pgmap v146: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-06-03T01:15:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:08 smithi107 ceph-mon[36622]: pgmap v146: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-06-03T01:15:09.079 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103576 2.04 MB/sec execute 76 sec latency 339.830 ms 2024-06-03T01:15:10.079 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103576 2.01 MB/sec execute 77 sec latency 1339.970 ms 2024-06-03T01:15:10.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:10 smithi094 ceph-mon[32119]: pgmap v147: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-06-03T01:15:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:10 smithi179 ceph-mon[37041]: pgmap v147: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-06-03T01:15:11.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:10 smithi107 ceph-mon[36622]: pgmap v147: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-06-03T01:15:11.079 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103595 1.99 MB/sec execute 78 sec latency 2307.300 ms 2024-06-03T01:15:12.079 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103632 1.96 MB/sec execute 79 sec latency 963.461 ms 2024-06-03T01:15:12.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:12 smithi094 ceph-mon[32119]: pgmap v148: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-06-03T01:15:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:12 smithi179 ceph-mon[37041]: pgmap v148: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-06-03T01:15:13.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:12 smithi107 ceph-mon[36622]: pgmap v148: 129 pgs: 129 active+clean; 47 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-06-03T01:15:13.079 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103632 1.94 MB/sec execute 80 sec latency 1963.623 ms 2024-06-03T01:15:14.080 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103632 1.91 MB/sec execute 81 sec latency 2963.786 ms 2024-06-03T01:15:14.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:14 smithi094 ceph-mon[32119]: pgmap v149: 129 pgs: 129 active+clean; 49 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-06-03T01:15:14.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:14 smithi179 ceph-mon[37041]: pgmap v149: 129 pgs: 129 active+clean; 49 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-06-03T01:15:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 -- 172.21.15.94:0/34844865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c98108860 msgr2=0x7f0c98114ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 --2- 172.21.15.94:0/34844865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c98108860 0x7f0c98114ca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009c10 tx=0x7f0c9002a6f0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 -- 172.21.15.94:0/34844865 shutdown_connections 2024-06-03T01:15:15.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 --2- 172.21.15.94:0/34844865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c98108860 0x7f0c98114ca0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 --2- 172.21.15.94:0/34844865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98107ee0 0x7f0c98108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 --2- 172.21.15.94:0/34844865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c9810de70 0x7f0c9810e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 -- 172.21.15.94:0/34844865 >> 172.21.15.94:0/34844865 conn(0x7f0c98077780 msgr2=0x7f0c981067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 -- 172.21.15.94:0/34844865 shutdown_connections 2024-06-03T01:15:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.017+0000 7f0c97200640 1 -- 172.21.15.94:0/34844865 wait complete. 2024-06-03T01:15:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.018+0000 7f0c97200640 1 Processor -- start 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.018+0000 7f0c97200640 1 -- start start 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c97200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c98107ee0 0x7f0c9819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c97200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98108860 0x7f0c9819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c97200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9810de70 0x7f0c981a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c97200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981181a0 con 0x7f0c98108860 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c97200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98118020 con 0x7f0c98107ee0 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c97200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98118320 con 0x7f0c9810de70 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98108860 0x7f0c9819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98108860 0x7f0c9819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35470/0 (socket says 172.21.15.94:35470) 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95400640 1 -- 172.21.15.94:0/4008692544 learned_addr learned my addr 172.21.15.94:0/4008692544 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95e00640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c98107ee0 0x7f0c9819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c96800640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9810de70 0x7f0c981a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95400640 1 -- 172.21.15.94:0/4008692544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9810de70 msgr2=0x7f0c981a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95400640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9810de70 0x7f0c981a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95400640 1 -- 172.21.15.94:0/4008692544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c98107ee0 msgr2=0x7f0c9819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.019+0000 7f0c95400640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c98107ee0 0x7f0c9819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c95400640 1 -- 172.21.15.94:0/4008692544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c981a3640 con 0x7f0c98108860 2024-06-03T01:15:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c96800640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9810de70 0x7f0c981a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c95400640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98108860 0x7f0c9819ebe0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400e9c0 tx=0x7f0c8400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c8ea00640 1 -- 172.21.15.94:0/4008692544 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8400cde0 con 0x7f0c98108860 2024-06-03T01:15:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c8ea00640 1 -- 172.21.15.94:0/4008692544 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c840035f0 con 0x7f0c98108860 2024-06-03T01:15:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c981a3930 con 0x7f0c98108860 2024-06-03T01:15:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c8ea00640 1 -- 172.21.15.94:0/4008692544 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84010690 con 0x7f0c98108860 2024-06-03T01:15:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.020+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c981ab1c0 con 0x7f0c98108860 2024-06-03T01:15:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.023+0000 7f0c9f9d2740 1 Processor -- start 2024-06-03T01:15:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.023+0000 7f0c8ea00640 1 -- 172.21.15.94:0/4008692544 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0c840037a0 con 0x7f0c98108860 2024-06-03T01:15:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c9f9d2740 1 -- start start 2024-06-03T01:15:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b417196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b417197680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b4170fb420 con 0x55b4171a1f80 2024-06-03T01:15:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b4170a2c50 con 0x55b417197200 2024-06-03T01:15:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b4170f73e0 con 0x55b417198770 2024-06-03T01:15:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c96800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b417196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c95400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c96800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b417196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49108/0 (socket says 172.21.15.94:49108) 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c96800640 1 -- 172.21.15.94:0/2161997968 learned_addr learned my addr 172.21.15.94:0/2161997968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.024+0000 7f0c95e00640 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b417197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c96800640 1 -- 172.21.15.94:0/2161997968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 msgr2=0x55b417197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c96800640 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b417197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c96800640 1 -- 172.21.15.94:0/2161997968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 msgr2=0x55b4171a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c96800640 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c96800640 1 -- 172.21.15.94:0/2161997968 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b416e0f210 con 0x55b417198770 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c8ea00640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0c7c0a2cf0 0x7f0c7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c8ea00640 1 -- 172.21.15.94:0/4008692544 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f0c840cfaa0 con 0x7f0c98108860 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c96800640 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b417196cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009c10 tx=0x7f0c90030de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6065664340a5837d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c95e00640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0c7c0a2cf0 0x7f0c7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.025+0000 7f0c83000640 1 -- 172.21.15.94:0/2161997968 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90041070 con 0x55b417198770 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.026+0000 7f0c83000640 1 -- 172.21.15.94:0/2161997968 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c90003e50 con 0x55b417198770 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.026+0000 7f0c83000640 1 -- 172.21.15.94:0/2161997968 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90036040 con 0x55b417198770 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.026+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/2161997968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 msgr2=0x55b417196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.026+0000 7f0c95e00640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0c7c0a2cf0 0x7f0c7c0a51b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55b417199ed0 tx=0x7f0c88005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.026+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b417196cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009c10 tx=0x7f0c90030de0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/2161997968 shutdown_connections 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b417197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/2161997968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b417196cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/2161997968 >> 172.21.15.94:0/2161997968 conn(0x55b41715ff80 msgr2=0x55b4171937f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/2161997968 shutdown_connections 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/2161997968 wait complete. 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 -- start start 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.027+0000 7f0c9f9d2740 1 Processor -- start 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b41719f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b41719f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b41719fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b4171a4cf0 con 0x55b417197200 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b4170b60b0 con 0x55b4171a1f80 2024-06-03T01:15:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b4171a4e70 con 0x55b417198770 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c9f9d2740 10 client.0 _ll_register_callbacks cb 0x55b4170a1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b41719f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b41719f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35492/0 (socket says 172.21.15.94:35492) 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95e00640 1 -- 172.21.15.94:0/1421666830 learned_addr learned my addr 172.21.15.94:0/1421666830 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c96800640 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b41719fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95400640 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b41719f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95e00640 1 -- 172.21.15.94:0/1421666830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 msgr2=0x55b41719f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95e00640 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b41719f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95e00640 1 -- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 msgr2=0x55b41719fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.028+0000 7f0c95e00640 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b41719fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c95e00640 1 -- 172.21.15.94:0/1421666830 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b416edb620 con 0x55b417197200 2024-06-03T01:15:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c95e00640 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b41719f000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c880045e0 tx=0x7f0c88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b416afa1ac0fd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c77400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c77400640 1 -- 172.21.15.94:0/1421666830 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88009e00 con 0x55b417197200 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c77400640 1 -- 172.21.15.94:0/1421666830 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f0c8801e680 con 0x55b417197200 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c90003e50 con 0x55b417197200 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c77400640 1 -- 172.21.15.94:0/1421666830 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0c8801e880 con 0x55b417197200 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c9f9d2740 10 client.15567 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.029+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b4171b1120 con 0x55b417197200 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c77400640 1 -- 172.21.15.94:0/1421666830 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c90003e50 con 0x55b417197200 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c77400640 1 -- 172.21.15.94:0/1421666830 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f0c88027070 con 0x55b417197200 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c9f9d2740 10 client.15567 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c9f9d2740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c9f9d2740 20 client.15567 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b4171b2630 0x55b4171b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c9f9d2740 4 client.15567 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:15:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.030+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b417108b50 con 0x55b4171b2630 2024-06-03T01:15:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.031+0000 7f0c95400640 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b4171b2630 0x55b4171b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.031+0000 7f0c95400640 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b4171b2630 0x55b4171b4a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58534a3827398bc server_cookie=d66565cfdc6c4a88 in_seq=0 out_seq=0 2024-06-03T01:15:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.031+0000 7f0c77400640 10 client.15567 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:14 smithi107 ceph-mon[36622]: pgmap v149: 129 pgs: 129 active+clean; 49 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-06-03T01:15:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c77400640 1 -- 172.21.15.94:0/1421666830 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b417108b50 con 0x55b4171b2630 2024-06-03T01:15:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c77400640 10 client.15567 handle_command_reply: tid=0 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 client.15567 shutdown 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 msgr2=0x55b41719f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b41719f000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c880045e0 tx=0x7f0c88004730 comp rx=0 tx=0).stop 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 shutdown_connections 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b4171b2630 0x55b4171b4a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b41719fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b41719f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/1421666830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b41719f000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c77400640 0 client.15567 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.032+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 >> 172.21.15.94:0/1421666830 conn(0x55b41715ff80 msgr2=0x55b417199660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.033+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 shutdown_connections 2024-06-03T01:15:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.033+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/1421666830 wait complete. 2024-06-03T01:15:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.033+0000 7f0c9f9d2740 20 client.15567 trim_cache size 0 max 16384 2024-06-03T01:15:15.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.049+0000 7f0c9f9d2740 1 Processor -- start 2024-06-03T01:15:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.049+0000 7f0c9f9d2740 1 -- start start 2024-06-03T01:15:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b4171994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981b51f0 con 0x55b4171a1f80 2024-06-03T01:15:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981b6d60 con 0x55b417197200 2024-06-03T01:15:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981b58c0 con 0x55b417198770 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171ba850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35502/0 (socket says 172.21.15.94:35502) 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 -- 172.21.15.94:0/3441363156 learned_addr learned my addr 172.21.15.94:0/3441363156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c95e00640 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b4171994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c95400640 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 -- 172.21.15.94:0/3441363156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 msgr2=0x55b4171ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 -- 172.21.15.94:0/3441363156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 msgr2=0x55b4171994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b4171994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c96800640 1 -- 172.21.15.94:0/3441363156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41718efd0 con 0x55b4171a1f80 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.050+0000 7f0c95e00640 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b4171994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.051+0000 7f0c96800640 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171ba850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c90004af0 tx=0x7f0c9002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c44b7ee760184e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.051+0000 7f0c82600640 1 -- 172.21.15.94:0/3441363156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90053070 con 0x55b4171a1f80 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.051+0000 7f0c82600640 1 -- 172.21.15.94:0/3441363156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c900045e0 con 0x55b4171a1f80 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.051+0000 7f0c82600640 1 -- 172.21.15.94:0/3441363156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90041040 con 0x55b4171a1f80 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/3441363156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 msgr2=0x55b4171ba850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171ba850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c90004af0 tx=0x7f0c9002a6f0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/3441363156 shutdown_connections 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b4171a1f80 0x55b4171ba850 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/3441363156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b417197200 0x55b4171994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/3441363156 >> 172.21.15.94:0/3441363156 conn(0x55b41715ff80 msgr2=0x55b417198dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/3441363156 shutdown_connections 2024-06-03T01:15:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/3441363156 wait complete. 2024-06-03T01:15:15.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 -- start start 2024-06-03T01:15:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.052+0000 7f0c9f9d2740 1 Processor -- start 2024-06-03T01:15:15.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b4171a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c9f9d2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b4171a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981b6690 con 0x55b417197200 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981b5f90 con 0x55b4171a1f80 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c9f9d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981b31d0 con 0x55b417198770 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c9f9d2740 10 client.0 _ll_register_callbacks cb 0x7f0c880044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b4171a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b4171a5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35506/0 (socket says 172.21.15.94:35506) 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c95e00640 1 -- 172.21.15.94:0/27231017 learned_addr learned my addr 172.21.15.94:0/27231017 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c96800640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b4171a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.053+0000 7f0c95400640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c96800640 1 -- 172.21.15.94:0/27231017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 msgr2=0x55b4171a64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c96800640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171a64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c96800640 1 -- 172.21.15.94:0/27231017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 msgr2=0x55b4171a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c96800640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b4171a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c96800640 1 -- 172.21.15.94:0/27231017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41721c390 con 0x55b4171a1f80 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c95e00640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b4171a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c96800640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b4171a69e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c90051040 tx=0x7f0c90009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1ddbe0f4276722 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c77400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c77400640 1 -- 172.21.15.94:0/27231017 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90053040 con 0x55b4171a1f80 2024-06-03T01:15:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c900045e0 con 0x55b4171a1f80 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.054+0000 7f0c77400640 1 -- 172.21.15.94:0/27231017 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f0c900303a0 con 0x55b4171a1f80 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c77400640 1 -- 172.21.15.94:0/27231017 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0c90041040 con 0x55b4171a1f80 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 10 client.25697 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b41721d680 con 0x55b4171a1f80 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c77400640 1 -- 172.21.15.94:0/27231017 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c900045e0 con 0x55b4171a1f80 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c77400640 1 -- 172.21.15.94:0/27231017 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f0c9003d460 con 0x55b4171a1f80 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 10 client.25697 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 20 client.25697 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b41721d8b0 0x55b41721fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 4 client.25697 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.055+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b417102ea0 con 0x55b41721d8b0 2024-06-03T01:15:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.056+0000 7f0c95e00640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b41721d8b0 0x55b41721fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.056+0000 7f0c95e00640 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b41721d8b0 0x55b41721fca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed7868ee4283e484 server_cookie=a9a12063fc294418 in_seq=0 out_seq=0 2024-06-03T01:15:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.056+0000 7f0c77400640 10 client.25697 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:15.080 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103632 1.89 MB/sec execute 82 sec latency 3963.991 ms 2024-06-03T01:15:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.098+0000 7f0c77400640 1 -- 172.21.15.94:0/27231017 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b417102ea0 con 0x55b41721d8b0 2024-06-03T01:15:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c77400640 10 client.25697 handle_command_reply: tid=0 2024-06-03T01:15:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 client.25697 shutdown 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 msgr2=0x55b4171a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b4171a69e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c90051040 tx=0x7f0c90009940 comp rx=0 tx=0).stop 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 shutdown_connections 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b41721d8b0 0x55b41721fca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b4171a1f80 0x55b4171a69e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b417198770 0x55b4171a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 --2- 172.21.15.94:0/27231017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b417197200 0x55b4171a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c77400640 0 client.25697 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 >> 172.21.15.94:0/27231017 conn(0x55b41715ff80 msgr2=0x55b417192970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 shutdown_connections 2024-06-03T01:15:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 1 -- 172.21.15.94:0/27231017 wait complete. 2024-06-03T01:15:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.099+0000 7f0c9f9d2740 20 client.25697 trim_cache size 0 max 16384 2024-06-03T01:15:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:15:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:15:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0c7c0a2cf0 msgr2=0x7f0c7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0c7c0a2cf0 0x7f0c7c0a51b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55b417199ed0 tx=0x7f0c88005f90 comp rx=0 tx=0).stop 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98108860 msgr2=0x7f0c9819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98108860 0x7f0c9819ebe0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400e9c0 tx=0x7f0c8400ee80 comp rx=0 tx=0).stop 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 shutdown_connections 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0c7c0a2cf0 0x7f0c7c0a51b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c9810de70 0x7f0c981a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c98108860 0x7f0c9819ebe0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 --2- 172.21.15.94:0/4008692544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c98107ee0 0x7f0c9819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 >> 172.21.15.94:0/4008692544 conn(0x7f0c98077780 msgr2=0x7f0c98078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 shutdown_connections 2024-06-03T01:15:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:15.100+0000 7f0c97200640 1 -- 172.21.15.94:0/4008692544 wait complete. 2024-06-03T01:15:16.080 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103649 1.87 MB/sec execute 83 sec latency 4935.423 ms 2024-06-03T01:15:16.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:16 smithi094 ceph-mon[32119]: pgmap v150: 129 pgs: 129 active+clean; 49 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 634 KiB/s wr, 115 op/s 2024-06-03T01:15:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:16 smithi179 ceph-mon[37041]: pgmap v150: 129 pgs: 129 active+clean; 49 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 634 KiB/s wr, 115 op/s 2024-06-03T01:15:17.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:16 smithi107 ceph-mon[36622]: pgmap v150: 129 pgs: 129 active+clean; 49 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 634 KiB/s wr, 115 op/s 2024-06-03T01:15:17.080 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103700 1.85 MB/sec execute 84 sec latency 945.763 ms 2024-06-03T01:15:18.080 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103700 1.82 MB/sec execute 85 sec latency 1945.920 ms 2024-06-03T01:15:18.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:18 smithi094 ceph-mon[32119]: pgmap v151: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 850 KiB/s wr, 141 op/s 2024-06-03T01:15:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:18 smithi179 ceph-mon[37041]: pgmap v151: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 850 KiB/s wr, 141 op/s 2024-06-03T01:15:19.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:18 smithi107 ceph-mon[36622]: pgmap v151: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 850 KiB/s wr, 141 op/s 2024-06-03T01:15:19.080 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103700 1.80 MB/sec execute 86 sec latency 2946.055 ms 2024-06-03T01:15:20.081 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103700 1.78 MB/sec execute 87 sec latency 3946.234 ms 2024-06-03T01:15:20.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:20 smithi094 ceph-mon[32119]: pgmap v152: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 406 KiB/s wr, 62 op/s 2024-06-03T01:15:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:20 smithi179 ceph-mon[37041]: pgmap v152: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 406 KiB/s wr, 62 op/s 2024-06-03T01:15:21.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:20 smithi107 ceph-mon[36622]: pgmap v152: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 406 KiB/s wr, 62 op/s 2024-06-03T01:15:21.081 INFO:tasks.workunit.client.0.smithi094.stdout: 1 103719 1.76 MB/sec execute 88 sec latency 4911.803 ms 2024-06-03T01:15:22.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:21 smithi107 ceph-mon[36622]: pgmap v153: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 406 KiB/s wr, 62 op/s 2024-06-03T01:15:22.081 INFO:tasks.workunit.client.0.smithi094.stdout: 1 105481 1.83 MB/sec execute 89 sec latency 16.092 ms 2024-06-03T01:15:22.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:21 smithi094 ceph-mon[32119]: pgmap v153: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 406 KiB/s wr, 62 op/s 2024-06-03T01:15:22.104 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:21 smithi179 ceph-mon[37041]: pgmap v153: 129 pgs: 129 active+clean; 47 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 406 KiB/s wr, 62 op/s 2024-06-03T01:15:23.081 INFO:tasks.workunit.client.0.smithi094.stdout: 1 106759 1.93 MB/sec execute 90 sec latency 19.904 ms 2024-06-03T01:15:24.081 INFO:tasks.workunit.client.0.smithi094.stdout: 1 107117 1.91 MB/sec execute 91 sec latency 449.834 ms 2024-06-03T01:15:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:23 smithi107 ceph-mon[36622]: pgmap v154: 129 pgs: 129 active+clean; 48 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 492 KiB/s wr, 80 op/s 2024-06-03T01:15:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:23 smithi094 ceph-mon[32119]: pgmap v154: 129 pgs: 129 active+clean; 48 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 492 KiB/s wr, 80 op/s 2024-06-03T01:15:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:23 smithi179 ceph-mon[37041]: pgmap v154: 129 pgs: 129 active+clean; 48 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 492 KiB/s wr, 80 op/s 2024-06-03T01:15:25.081 INFO:tasks.workunit.client.0.smithi094.stdout: 1 107117 1.89 MB/sec execute 92 sec latency 1449.978 ms 2024-06-03T01:15:26.081 INFO:tasks.workunit.client.0.smithi094.stdout: 1 107136 1.87 MB/sec execute 93 sec latency 2416.780 ms 2024-06-03T01:15:26.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:25 smithi107 ceph-mon[36622]: pgmap v155: 129 pgs: 129 active+clean; 48 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 303 KiB/s wr, 44 op/s 2024-06-03T01:15:26.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:25 smithi094 ceph-mon[32119]: pgmap v155: 129 pgs: 129 active+clean; 48 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 303 KiB/s wr, 44 op/s 2024-06-03T01:15:26.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:25 smithi179 ceph-mon[37041]: pgmap v155: 129 pgs: 129 active+clean; 48 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 303 KiB/s wr, 44 op/s 2024-06-03T01:15:27.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 107241 1.85 MB/sec execute 94 sec latency 867.630 ms 2024-06-03T01:15:28.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 107241 1.83 MB/sec execute 95 sec latency 1867.773 ms 2024-06-03T01:15:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 -- 172.21.15.94:0/608708625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 msgr2=0x7fc688110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 --2- 172.21.15.94:0/608708625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc688110ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc680009ca0 tx=0x7fc68002a8c0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 -- 172.21.15.94:0/608708625 shutdown_connections 2024-06-03T01:15:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 --2- 172.21.15.94:0/608708625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc688110ab0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 --2- 172.21.15.94:0/608708625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6881047c0 0x7fc688108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 --2- 172.21.15.94:0/608708625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc688103e10 0x7fc6881041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 -- 172.21.15.94:0/608708625 >> 172.21.15.94:0/608708625 conn(0x7fc6880ff6d0 msgr2=0x7fc688101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 -- 172.21.15.94:0/608708625 shutdown_connections 2024-06-03T01:15:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.241+0000 7fc68ce00640 1 -- 172.21.15.94:0/608708625 wait complete. 2024-06-03T01:15:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.242+0000 7fc68ce00640 1 Processor -- start 2024-06-03T01:15:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.242+0000 7fc68ce00640 1 -- start start 2024-06-03T01:15:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc68ce00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc688103e10 0x7fc68819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc68ce00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc68819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc68ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc68819ec70 0x7fc68819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc68ce00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc688113f90 con 0x7fc688108ee0 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc68ce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc688113e10 con 0x7fc68819ec70 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc68ce00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc688114110 con 0x7fc688103e10 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc68819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc68819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35458/0 (socket says 172.21.15.94:35458) 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685400640 1 -- 172.21.15.94:0/295378115 learned_addr learned my addr 172.21.15.94:0/295378115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685e00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc688103e10 0x7fc68819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc686800640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc68819ec70 0x7fc68819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685e00640 1 -- 172.21.15.94:0/295378115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc68819ec70 msgr2=0x7fc68819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685e00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc68819ec70 0x7fc68819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685e00640 1 -- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 msgr2=0x7fc68819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685e00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc68819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685e00640 1 -- 172.21.15.94:0/295378115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68819f760 con 0x7fc688103e10 2024-06-03T01:15:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.243+0000 7fc685400640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc68819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.244+0000 7fc685e00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc688103e10 0x7fc68819a450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc67800e9f0 tx=0x7fc67800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.244+0000 7fc67ea00640 1 -- 172.21.15.94:0/295378115 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67800ccc0 con 0x7fc688103e10 2024-06-03T01:15:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.244+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68819fa50 con 0x7fc688103e10 2024-06-03T01:15:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.244+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6881a71e0 con 0x7fc688103e10 2024-06-03T01:15:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.244+0000 7fc67ea00640 1 -- 172.21.15.94:0/295378115 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc678005020 con 0x7fc688103e10 2024-06-03T01:15:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.244+0000 7fc67ea00640 1 -- 172.21.15.94:0/295378115 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67801c070 con 0x7fc688103e10 2024-06-03T01:15:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.247+0000 7fc68fbf6740 1 Processor -- start 2024-06-03T01:15:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.247+0000 7fc68fbf6740 1 -- start start 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6bfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6c0040 0x55f61f6c04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6cadc0 0x55f61f6cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f61f624420 con 0x55f61f6cadc0 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f61f5cbc50 con 0x55f61f6c15b0 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f61f6203e0 con 0x55f61f6c0040 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc67ea00640 1 -- 172.21.15.94:0/295378115 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc678004a00 con 0x7fc688103e10 2024-06-03T01:15:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6bfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6bfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40660/0 (socket says 172.21.15.94:40660) 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc685400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6cadc0 0x55f61f6cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 -- 172.21.15.94:0/511103937 learned_addr learned my addr 172.21.15.94:0/511103937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc685e00640 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6c0040 0x55f61f6c04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 -- 172.21.15.94:0/511103937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6c0040 msgr2=0x55f61f6c04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6c0040 0x55f61f6c04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 -- 172.21.15.94:0/511103937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6cadc0 msgr2=0x55f61f6cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6cadc0 0x55f61f6cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc686800640 1 -- 172.21.15.94:0/511103937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61f338210 con 0x55f61f6c15b0 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.248+0000 7fc685e00640 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6c0040 0x55f61f6c04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.249+0000 7fc686800640 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6bfb00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc680035000 tx=0x7fc680005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4193900377925b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.249+0000 7fc673400640 1 -- 172.21.15.94:0/511103937 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680031db0 con 0x55f61f6c15b0 2024-06-03T01:15:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.249+0000 7fc673400640 1 -- 172.21.15.94:0/511103937 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc680036040 con 0x55f61f6c15b0 2024-06-03T01:15:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.249+0000 7fc673400640 1 -- 172.21.15.94:0/511103937 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680004340 con 0x55f61f6c15b0 2024-06-03T01:15:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.249+0000 7fc67ea00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc66c0a2e20 0x7fc66c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 -- 172.21.15.94:0/511103937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 msgr2=0x55f61f6bfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6bfb00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc680035000 tx=0x7fc680005f10 comp rx=0 tx=0).stop 2024-06-03T01:15:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc67ea00640 1 -- 172.21.15.94:0/295378115 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fc6780cf630 con 0x7fc688103e10 2024-06-03T01:15:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc685e00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc66c0a2e20 0x7fc66c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 -- 172.21.15.94:0/511103937 shutdown_connections 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6cadc0 0x55f61f6cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6c0040 0x55f61f6c04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/511103937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6bfb00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 -- 172.21.15.94:0/511103937 >> 172.21.15.94:0/511103937 conn(0x55f61f688f80 msgr2=0x55f61f6bc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 -- 172.21.15.94:0/511103937 shutdown_connections 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 -- 172.21.15.94:0/511103937 wait complete. 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc685e00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc66c0a2e20 0x7fc66c0a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55f61f6c2d10 tx=0x7fc67800cf30 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.250+0000 7fc68fbf6740 1 -- start start 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 1 Processor -- start 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c0040 0x55f61f6c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f61f6cd9b0 con 0x55f61f6c0040 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f61f5df0b0 con 0x55f61f6c15b0 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f61f6cdb30 con 0x55f61f6cadc0 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc68fbf6740 10 client.0 _ll_register_callbacks cb 0x55f61f5caf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.251+0000 7fc686800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c0040 0x55f61f6c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c0040 0x55f61f6c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35472/0 (socket says 172.21.15.94:35472) 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 -- 172.21.15.94:0/3083747316 learned_addr learned my addr 172.21.15.94:0/3083747316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc685400640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 -- 172.21.15.94:0/3083747316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 msgr2=0x55f61f6c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 -- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 msgr2=0x55f61f6c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 -- 172.21.15.94:0/3083747316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61f404620 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc685400640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc686800640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c0040 0x55f61f6c7de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc680009620 tx=0x7fc680031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd51b7842a9ae18 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.252+0000 7fc667e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc667e00640 1 -- 172.21.15.94:0/3083747316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680051060 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc667e00640 1 -- 172.21.15.94:0/3083747316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fc6800040c0 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc680036040 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc667e00640 1 -- 172.21.15.94:0/3083747316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc680003e40 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 10 client.15585 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f61f6cd9b0 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc667e00640 1 -- 172.21.15.94:0/3083747316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc680036040 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc667e00640 1 -- 172.21.15.94:0/3083747316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7fc680043020 con 0x55f61f6c0040 2024-06-03T01:15:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 10 client.15585 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 20 client.15585 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f6daf40 0x55f61f6dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 4 client.15585 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.253+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61f631b50 con 0x55f61f6daf40 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.254+0000 7fc685400640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f6daf40 0x55f61f6dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.254+0000 7fc685400640 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f6daf40 0x55f61f6dd350 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e68c79b888796a server_cookie=b5de99ddfb17dbea in_seq=0 out_seq=0 2024-06-03T01:15:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.254+0000 7fc667e00640 10 client.15585 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc667e00640 1 -- 172.21.15.94:0/3083747316 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f61f631b50 con 0x55f61f6daf40 2024-06-03T01:15:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc667e00640 10 client.15585 handle_command_reply: tid=0 2024-06-03T01:15:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc68fbf6740 1 client.15585 shutdown 2024-06-03T01:15:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c0040 msgr2=0x55f61f6c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c0040 0x55f61f6c7de0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc680009620 tx=0x7fc680031a60 comp rx=0 tx=0).stop 2024-06-03T01:15:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 shutdown_connections 2024-06-03T01:15:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.255+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f6daf40 0x55f61f6dd350 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.256+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c15b0 0x55f61f6c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.256+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3083747316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c0040 0x55f61f6c7de0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.256+0000 7fc667e00640 0 client.15585 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.256+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 >> 172.21.15.94:0/3083747316 conn(0x55f61f688f80 msgr2=0x55f61f6c24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.256+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 shutdown_connections 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.256+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3083747316 wait complete. 2024-06-03T01:15:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.256+0000 7fc68fbf6740 20 client.15585 trim_cache size 0 max 16384 2024-06-03T01:15:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 Processor -- start 2024-06-03T01:15:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 -- start start 2024-06-03T01:15:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6bbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc66800fdf0 con 0x55f61f6c15b0 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc668011960 con 0x55f61f6c0040 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc6680104c0 con 0x55f61f6cadc0 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.273+0000 7fc685400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6c2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35482/0 (socket says 172.21.15.94:35482) 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 -- 172.21.15.94:0/2115296501 learned_addr learned my addr 172.21.15.94:0/2115296501 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc686800640 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc686800640 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6bbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 -- 172.21.15.94:0/2115296501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 msgr2=0x55f61f6bbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6bbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 -- 172.21.15.94:0/2115296501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 msgr2=0x55f61f6c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 -- 172.21.15.94:0/2115296501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61f6bfe70 con 0x55f61f6c15b0 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc686800640 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc685400640 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6c2320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc6740058c0 tx=0x7fc674005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92f4a461ab004f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc672a00640 1 -- 172.21.15.94:0/2115296501 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67400fcb0 con 0x55f61f6c15b0 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc672a00640 1 -- 172.21.15.94:0/2115296501 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67400fe50 con 0x55f61f6c15b0 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.274+0000 7fc672a00640 1 -- 172.21.15.94:0/2115296501 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674004a00 con 0x55f61f6c15b0 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 -- 172.21.15.94:0/2115296501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 msgr2=0x55f61f6c2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6c2320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc6740058c0 tx=0x7fc674005f90 comp rx=0 tx=0).stop 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 -- 172.21.15.94:0/2115296501 shutdown_connections 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f6bbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6c2320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/2115296501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 -- 172.21.15.94:0/2115296501 >> 172.21.15.94:0/2115296501 conn(0x55f61f688f80 msgr2=0x55f61f6c1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 -- 172.21.15.94:0/2115296501 shutdown_connections 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.275+0000 7fc68fbf6740 1 -- 172.21.15.94:0/2115296501 wait complete. 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 -- start start 2024-06-03T01:15:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 Processor -- start 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f743be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc668011290 con 0x55f61f6c15b0 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc668010b90 con 0x55f61f6c0040 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc66800ddd0 con 0x55f61f6cadc0 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc68fbf6740 10 client.0 _ll_register_callbacks cb 0x7fc674009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.276+0000 7fc685400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35486/0 (socket says 172.21.15.94:35486) 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 -- 172.21.15.94:0/3479091452 learned_addr learned my addr 172.21.15.94:0/3479091452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc686800640 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc686800640 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f743be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 -- 172.21.15.94:0/3479091452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 msgr2=0x55f61f743be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f743be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 -- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 msgr2=0x55f61f6cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 -- 172.21.15.94:0/3479091452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61f6cec60 con 0x55f61f6c15b0 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc685400640 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6ce3d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc674005510 tx=0x7fc67400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfc756db114bf60 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc667e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:15:28.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.277+0000 7fc667e00640 1 -- 172.21.15.94:0/3479091452 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674004230 con 0x55f61f6c15b0 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc67400fe50 con 0x55f61f6c15b0 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc667e00640 1 -- 172.21.15.94:0/3479091452 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fc6740043d0 con 0x55f61f6c15b0 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc667e00640 1 -- 172.21.15.94:0/3479091452 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc674036050 con 0x55f61f6c15b0 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 10 client.15591 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f61f745200 con 0x55f61f6c15b0 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc667e00640 1 -- 172.21.15.94:0/3479091452 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67400fe50 con 0x55f61f6c15b0 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc667e00640 1 -- 172.21.15.94:0/3479091452 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7fc67402a680 con 0x55f61f6c15b0 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 10 client.15591 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 20 client.15591 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f745840 0x55f61f747c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 4 client.15591 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.278+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f61f62bea0 con 0x55f61f745840 2024-06-03T01:15:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.279+0000 7fc686800640 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f745840 0x55f61f747c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.279+0000 7fc686800640 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f745840 0x55f61f747c60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef668fad2f2ed6b5 server_cookie=30e35d3a7a70e6c in_seq=0 out_seq=0 2024-06-03T01:15:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.279+0000 7fc667e00640 10 client.15591 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:28.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:27 smithi107 ceph-mon[36622]: pgmap v156: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 482 KiB/s wr, 76 op/s 2024-06-03T01:15:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc667e00640 1 -- 172.21.15.94:0/3479091452 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fc680049030 con 0x55f61f745840 2024-06-03T01:15:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc667e00640 10 client.15591 handle_command_reply: tid=0 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 client.15591 shutdown 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 msgr2=0x55f61f6ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6ce3d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc674005510 tx=0x7fc67400fcd0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 shutdown_connections 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f61f6cadc0 0x55f61f743be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f61f6c15b0 0x55f61f6ce3d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f61f745840 0x55f61f747c60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 --2- 172.21.15.94:0/3479091452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61f6c0040 0x55f61f6cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc667e00640 0 client.15591 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 >> 172.21.15.94:0/3479091452 conn(0x55f61f688f80 msgr2=0x55f61f6c24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 shutdown_connections 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 1 -- 172.21.15.94:0/3479091452 wait complete. 2024-06-03T01:15:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.310+0000 7fc68fbf6740 20 client.15591 trim_cache size 0 max 16384 2024-06-03T01:15:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:15:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:15:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:15:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc66c0a2e20 msgr2=0x7fc66c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc66c0a2e20 0x7fc66c0a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55f61f6c2d10 tx=0x7fc67800cf30 comp rx=0 tx=0).stop 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc688103e10 msgr2=0x7fc68819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc688103e10 0x7fc68819a450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc67800e9f0 tx=0x7fc67800eeb0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 shutdown_connections 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc66c0a2e20 0x7fc66c0a52e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc68819ec70 0x7fc68819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc688108ee0 0x7fc68819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 --2- 172.21.15.94:0/295378115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc688103e10 0x7fc68819a450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 >> 172.21.15.94:0/295378115 conn(0x7fc6880ff6d0 msgr2=0x7fc688101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 shutdown_connections 2024-06-03T01:15:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:28.311+0000 7fc68ce00640 1 -- 172.21.15.94:0/295378115 wait complete. 2024-06-03T01:15:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:27 smithi094 ceph-mon[32119]: pgmap v156: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 482 KiB/s wr, 76 op/s 2024-06-03T01:15:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:27 smithi179 ceph-mon[37041]: pgmap v156: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 482 KiB/s wr, 76 op/s 2024-06-03T01:15:29.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 107241 1.81 MB/sec execute 96 sec latency 2867.871 ms 2024-06-03T01:15:30.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 107241 1.80 MB/sec execute 97 sec latency 3867.970 ms 2024-06-03T01:15:30.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:30 smithi094 ceph-mon[32119]: pgmap v157: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 265 KiB/s wr, 50 op/s 2024-06-03T01:15:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:30 smithi179 ceph-mon[37041]: pgmap v157: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 265 KiB/s wr, 50 op/s 2024-06-03T01:15:31.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:30 smithi107 ceph-mon[36622]: pgmap v157: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 265 KiB/s wr, 50 op/s 2024-06-03T01:15:31.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 108556 1.83 MB/sec execute 98 sec latency 4153.590 ms 2024-06-03T01:15:32.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110086 1.93 MB/sec execute 99 sec latency 13.527 ms 2024-06-03T01:15:32.834 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:32 smithi107 ceph-mon[36622]: pgmap v158: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 265 KiB/s wr, 50 op/s 2024-06-03T01:15:32.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:32 smithi094 ceph-mon[32119]: pgmap v158: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 265 KiB/s wr, 50 op/s 2024-06-03T01:15:32.854 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:32 smithi179 ceph-mon[37041]: pgmap v158: 129 pgs: 129 active+clean; 50 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 265 KiB/s wr, 50 op/s 2024-06-03T01:15:33.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110722 1.93 MB/sec execute 100 sec latency 16.881 ms 2024-06-03T01:15:34.082 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110723 1.91 MB/sec execute 101 sec latency 985.723 ms 2024-06-03T01:15:34.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:34 smithi094 ceph-mon[32119]: pgmap v159: 129 pgs: 129 active+clean; 51 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-03T01:15:34.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:34 smithi179 ceph-mon[37041]: pgmap v159: 129 pgs: 129 active+clean; 51 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-03T01:15:35.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:34 smithi107 ceph-mon[36622]: pgmap v159: 129 pgs: 129 active+clean; 51 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-03T01:15:35.083 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110723 1.89 MB/sec execute 102 sec latency 1985.867 ms 2024-06-03T01:15:36.083 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110752 1.88 MB/sec execute 103 sec latency 2950.708 ms 2024-06-03T01:15:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:36 smithi094 ceph-mon[32119]: pgmap v160: 129 pgs: 129 active+clean; 51 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-06-03T01:15:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:36 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:36 smithi179 ceph-mon[37041]: pgmap v160: 129 pgs: 129 active+clean; 51 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-06-03T01:15:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:36 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:37.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:36 smithi107 ceph-mon[36622]: pgmap v160: 129 pgs: 129 active+clean; 51 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-06-03T01:15:37.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:36 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:15:37.083 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110791 1.86 MB/sec execute 104 sec latency 966.507 ms 2024-06-03T01:15:37.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:37 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:37.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:37 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:37.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:37 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:15:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:37 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:37 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:37 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:15:38.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:37 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:15:38.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:37 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:15:38.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:37 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:15:38.083 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110791 1.84 MB/sec execute 105 sec latency 1966.591 ms 2024-06-03T01:15:38.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:38 smithi094 ceph-mon[32119]: pgmap v161: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-06-03T01:15:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:38 smithi179 ceph-mon[37041]: pgmap v161: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-06-03T01:15:39.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:38 smithi107 ceph-mon[36622]: pgmap v161: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-06-03T01:15:39.499 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110791 1.82 MB/sec execute 106 sec latency 2966.680 ms 2024-06-03T01:15:39.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:39 smithi094 ceph-mon[32119]: pgmap v162: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-06-03T01:15:39.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:39 smithi179 ceph-mon[37041]: pgmap v162: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-06-03T01:15:40.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:39 smithi107 ceph-mon[36622]: pgmap v162: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-06-03T01:15:40.083 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110791 1.81 MB/sec execute 107 sec latency 3966.842 ms 2024-06-03T01:15:41.083 INFO:tasks.workunit.client.0.smithi094.stdout: 1 110813 1.79 MB/sec execute 108 sec latency 4931.830 ms 2024-06-03T01:15:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 -- 172.21.15.94:0/3090373952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580103e10 msgr2=0x7f25801041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 --2- 172.21.15.94:0/3090373952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580103e10 0x7f25801041f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2574009900 tx=0x7f257402a410 comp rx=0 tx=0).stop 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 -- 172.21.15.94:0/3090373952 shutdown_connections 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 --2- 172.21.15.94:0/3090373952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580108ee0 0x7f2580110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 --2- 172.21.15.94:0/3090373952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25801047c0 0x7f2580108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 --2- 172.21.15.94:0/3090373952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2580103e10 0x7f25801041f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 -- 172.21.15.94:0/3090373952 >> 172.21.15.94:0/3090373952 conn(0x7f25800ff6d0 msgr2=0x7f2580101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 -- 172.21.15.94:0/3090373952 shutdown_connections 2024-06-03T01:15:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.454+0000 7f2587800640 1 -- 172.21.15.94:0/3090373952 wait complete. 2024-06-03T01:15:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.455+0000 7f2587800640 1 Processor -- start 2024-06-03T01:15:41.457 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2587800640 1 -- start start 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2587800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580103e10 0x7f258019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2587800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2580108ee0 0x7f258019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2587800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f258019ed20 0x7f258019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2587800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2580113f90 con 0x7f2580103e10 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2587800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2580113e10 con 0x7f2580108ee0 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2587800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2580114110 con 0x7f258019ed20 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2584e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580103e10 0x7f258019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f257fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2580108ee0 0x7f258019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2584e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580103e10 0x7f258019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42924/0 (socket says 172.21.15.94:42924) 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f257fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2580108ee0 0x7f258019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42720/0 (socket says 172.21.15.94:42720) 2024-06-03T01:15:41.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2584e00640 1 -- 172.21.15.94:0/794608290 learned_addr learned my addr 172.21.15.94:0/794608290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.456+0000 7f2585800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f258019ed20 0x7f258019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257fe00640 1 -- 172.21.15.94:0/794608290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f258019ed20 msgr2=0x7f258019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257fe00640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f258019ed20 0x7f258019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257fe00640 1 -- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580103e10 msgr2=0x7f258019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257fe00640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580103e10 0x7f258019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257fe00640 1 -- 172.21.15.94:0/794608290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258019f810 con 0x7f2580108ee0 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f2585800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f258019ed20 0x7f258019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f2584e00640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580103e10 0x7f258019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257fe00640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2580108ee0 0x7f258019a9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f256c00e9c0 tx=0x7f256c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257d600640 1 -- 172.21.15.94:0/794608290 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c00ccc0 con 0x7f2580108ee0 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f257d600640 1 -- 172.21.15.94:0/794608290 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f256c005020 con 0x7f2580108ee0 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.457+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258019fb00 con 0x7f2580108ee0 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.458+0000 7f257d600640 1 -- 172.21.15.94:0/794608290 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c004a00 con 0x7f2580108ee0 2024-06-03T01:15:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.458+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25801a7210 con 0x7f2580108ee0 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 Processor -- start 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 -- start start 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f257d600640 1 -- 172.21.15.94:0/794608290 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f256c0051d0 con 0x7f2580108ee0 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e1fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e204c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e2adc0 0x562b12e2d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b12d84420 con 0x562b12e2adc0 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b12d2bc50 con 0x562b12e20040 2024-06-03T01:15:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.461+0000 7f258a5ef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b12d803e0 con 0x562b12e215b0 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2585800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e1fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e204c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42736/0 (socket says 172.21.15.94:42736) 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 -- 172.21.15.94:0/1717772007 learned_addr learned my addr 172.21.15.94:0/1717772007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f257fe00640 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e2adc0 0x562b12e2d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 -- 172.21.15.94:0/1717772007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 msgr2=0x562b12e1fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e1fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 -- 172.21.15.94:0/1717772007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e2adc0 msgr2=0x562b12e2d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e2adc0 0x562b12e2d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 -- 172.21.15.94:0/1717772007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b12a98210 con 0x562b12e20040 2024-06-03T01:15:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f257fe00640 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e2adc0 0x562b12e2d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.462+0000 7f2584e00640 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e204c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2574005dd0 tx=0x7f2574005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c8adb566e8cf3c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.463+0000 7f2572000640 1 -- 172.21.15.94:0/1717772007 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574035040 con 0x562b12e20040 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.463+0000 7f2572000640 1 -- 172.21.15.94:0/1717772007 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25740319f0 con 0x562b12e20040 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.463+0000 7f257d600640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f25680a2da0 0x7f25680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.463+0000 7f2572000640 1 -- 172.21.15.94:0/1717772007 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740049c0 con 0x562b12e20040 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.463+0000 7f257d600640 1 -- 172.21.15.94:0/794608290 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f256c0cf770 con 0x7f2580108ee0 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.463+0000 7f2585800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f25680a2da0 0x7f25680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 -- 172.21.15.94:0/1717772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 msgr2=0x562b12e204c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e204c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2574005dd0 tx=0x7f2574005e00 comp rx=0 tx=0).stop 2024-06-03T01:15:41.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 -- 172.21.15.94:0/1717772007 shutdown_connections 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e2adc0 0x562b12e2d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e204c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1717772007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e1fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 -- 172.21.15.94:0/1717772007 >> 172.21.15.94:0/1717772007 conn(0x562b12de8f80 msgr2=0x562b12e1c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 -- 172.21.15.94:0/1717772007 shutdown_connections 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f2585800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f25680a2da0 0x7f25680a5260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2578009fd0 tx=0x7f2578009500 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 -- 172.21.15.94:0/1717772007 wait complete. 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.464+0000 7f258a5ef740 1 -- start start 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 1 Processor -- start 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e28370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e2adc0 0x562b12e288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562b12e2db30 con 0x562b12e20040 2024-06-03T01:15:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562b12d3f0b0 con 0x562b12e215b0 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b12e2dcb0 con 0x562b12e2adc0 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f258a5ef740 10 client.0 _ll_register_callbacks cb 0x562b12d2af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f2585800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e2adc0 0x562b12e288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f257fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e28370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f2584e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f2585800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e2adc0 0x562b12e288b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:40608/0 (socket says 172.21.15.94:40608) 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f2585800640 1 -- 172.21.15.94:0/3210886044 learned_addr learned my addr 172.21.15.94:0/3210886044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2585800640 1 -- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 msgr2=0x562b12e28370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.465+0000 7f257fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e28370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42744/0 (socket says 172.21.15.94:42744) 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2585800640 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e28370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2585800640 1 -- 172.21.15.94:0/3210886044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 msgr2=0x562b12e27e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2585800640 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e27e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2585800640 1 -- 172.21.15.94:0/3210886044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b12b64620 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2584e00640 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f257fe00640 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e28370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2585800640 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e2adc0 0x562b12e288b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2578013010 tx=0x7f2578004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c126558272e03f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.466+0000 7f2566a00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.467+0000 7f2566a00640 1 -- 172.21.15.94:0/3210886044 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25780045b0 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.467+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25740319f0 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.467+0000 7f2566a00640 1 -- 172.21.15.94:0/3210886044 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f2578004750 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.467+0000 7f2566a00640 1 -- 172.21.15.94:0/3210886044 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2578012020 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.467+0000 7f258a5ef740 10 client.25738 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.467+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562b12e39f10 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.467+0000 7f2566a00640 1 -- 172.21.15.94:0/3210886044 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25740319f0 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f2566a00640 1 -- 172.21.15.94:0/3210886044 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f2578027070 con 0x562b12e2adc0 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f258a5ef740 10 client.25738 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f258a5ef740 10 client.25738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f258a5ef740 20 client.25738 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e3b3d0 0x562b12e3d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f258a5ef740 4 client.25738 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:15:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b12d91b50 con 0x562b12e3b3d0 2024-06-03T01:15:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.468+0000 7f2584e00640 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e3b3d0 0x562b12e3d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.469+0000 7f2584e00640 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e3b3d0 0x562b12e3d7e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31958465d7f92da5 server_cookie=60111b810606a113 in_seq=0 out_seq=0 2024-06-03T01:15:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.469+0000 7f2566a00640 10 client.25738 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.469+0000 7f2566a00640 1 -- 172.21.15.94:0/3210886044 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b12d91b50 con 0x562b12e3b3d0 2024-06-03T01:15:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.469+0000 7f2566a00640 10 client.25738 handle_command_reply: tid=0 2024-06-03T01:15:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.469+0000 7f258a5ef740 1 client.25738 shutdown 2024-06-03T01:15:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e2adc0 msgr2=0x562b12e288b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e2adc0 0x562b12e288b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2578013010 tx=0x7f2578004ab0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 shutdown_connections 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e2adc0 0x562b12e288b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e3b3d0 0x562b12e3d7e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e28370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3210886044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f2566a00640 0 client.25738 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 >> 172.21.15.94:0/3210886044 conn(0x562b12de8f80 msgr2=0x562b12e224a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 shutdown_connections 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 1 -- 172.21.15.94:0/3210886044 wait complete. 2024-06-03T01:15:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.470+0000 7f258a5ef740 20 client.25738 trim_cache size 0 max 16384 2024-06-03T01:15:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 Processor -- start 2024-06-03T01:15:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 -- start start 2024-06-03T01:15:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12de8f80 0x562b12e222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e1bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e1c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b0f30 con 0x562b12e20040 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b2aa0 con 0x562b12e215b0 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f258a5ef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b1600 con 0x562b12de8f80 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f257fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e1bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f2585800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e1c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f2584e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12de8f80 0x562b12e222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f2585800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e1c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42752/0 (socket says 172.21.15.94:42752) 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f2584e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12de8f80 0x562b12e222b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:40624/0 (socket says 172.21.15.94:40624) 2024-06-03T01:15:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f257fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e1bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42962/0 (socket says 172.21.15.94:42962) 2024-06-03T01:15:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.488+0000 7f2585800640 1 -- 172.21.15.94:0/3310001747 learned_addr learned my addr 172.21.15.94:0/3310001747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2585800640 1 -- 172.21.15.94:0/3310001747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12de8f80 msgr2=0x562b12e222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2585800640 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12de8f80 0x562b12e222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2585800640 1 -- 172.21.15.94:0/3310001747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 msgr2=0x562b12e1bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2585800640 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e1bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2585800640 1 -- 172.21.15.94:0/3310001747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b12e20b10 con 0x562b12e215b0 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2584e00640 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12de8f80 0x562b12e222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2585800640 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e1c4d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2578011590 tx=0x7f25780127d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1dcc560a2d4d726 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2571600640 1 -- 172.21.15.94:0/3310001747 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578004400 con 0x562b12e215b0 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2571600640 1 -- 172.21.15.94:0/3310001747 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2578011780 con 0x562b12e215b0 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.489+0000 7f2571600640 1 -- 172.21.15.94:0/3310001747 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257801e6d0 con 0x562b12e215b0 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 -- 172.21.15.94:0/3310001747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 msgr2=0x562b12e1c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e1c4d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2578011590 tx=0x7f25780127d0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 -- 172.21.15.94:0/3310001747 shutdown_connections 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e215b0 0x562b12e1c4d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12e20040 0x562b12e1bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 --2- 172.21.15.94:0/3310001747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12de8f80 0x562b12e222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 -- 172.21.15.94:0/3310001747 >> 172.21.15.94:0/3310001747 conn(0x562b12e17bc0 msgr2=0x562b12e21ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 -- 172.21.15.94:0/3310001747 shutdown_connections 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 -- 172.21.15.94:0/3310001747 wait complete. 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.490+0000 7f258a5ef740 1 -- start start 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 1 Processor -- start 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12de8f80 0x562b12e333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e24960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e24ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b23d0 con 0x562b12de8f80 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b1cd0 con 0x562b12e20040 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f25801aef10 con 0x562b12e215b0 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f258a5ef740 10 client.0 _ll_register_callbacks cb 0x7f25780125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f257fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e24960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f2585800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e24ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f2584e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12de8f80 0x562b12e333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f257fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e24960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42762/0 (socket says 172.21.15.94:42762) 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f2585800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e24ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:40628/0 (socket says 172.21.15.94:40628) 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.491+0000 7f257fe00640 1 -- 172.21.15.94:0/1996265040 learned_addr learned my addr 172.21.15.94:0/1996265040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f257fe00640 1 -- 172.21.15.94:0/1996265040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 msgr2=0x562b12e24ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f257fe00640 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e24ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f257fe00640 1 -- 172.21.15.94:0/1996265040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12de8f80 msgr2=0x562b12e333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f257fe00640 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12de8f80 0x562b12e333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f257fe00640 1 -- 172.21.15.94:0/1996265040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b12e253e0 con 0x562b12e20040 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f2585800640 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e24ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f257fe00640 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e24960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f256c097d30 tx=0x7f256c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec50ecb9b88b1ea8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f2566a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f2566a00640 1 -- 172.21.15.94:0/1996265040 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c01c050 con 0x562b12e20040 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f2566a00640 1 -- 172.21.15.94:0/1996265040 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f256c0ccd80 con 0x562b12e20040 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.492+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2578011780 con 0x562b12e20040 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f2566a00640 1 -- 172.21.15.94:0/1996265040 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f256c004e60 con 0x562b12e20040 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 10 client.25730 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562b12ea5e20 con 0x562b12e20040 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f2566a00640 1 -- 172.21.15.94:0/1996265040 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2578011780 con 0x562b12e20040 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f2566a00640 1 -- 172.21.15.94:0/1996265040 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f256c0cf330 con 0x562b12e20040 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 10 client.25730 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 20 client.25730 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e2adc0 0x562b12ea8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 4 client.25730 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:15:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b12d8bea0 con 0x562b12e2adc0 2024-06-03T01:15:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.493+0000 7f2584e00640 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e2adc0 0x562b12ea8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.494+0000 7f2584e00640 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e2adc0 0x562b12ea8600 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e4390de72d0e5e server_cookie=7093db46b3f43199 in_seq=0 out_seq=0 2024-06-03T01:15:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.494+0000 7f2566a00640 10 client.25730 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.521+0000 7f2566a00640 1 -- 172.21.15.94:0/1996265040 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b12d8bea0 con 0x562b12e2adc0 2024-06-03T01:15:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.521+0000 7f2566a00640 10 client.25730 handle_command_reply: tid=0 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 client.25730 shutdown 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 msgr2=0x562b12e24960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e24960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f256c097d30 tx=0x7f256c009f60 comp rx=0 tx=0).stop 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 shutdown_connections 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b12e215b0 0x562b12e24ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562b12e2adc0 0x562b12ea8600 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b12e20040 0x562b12e24960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 --2- 172.21.15.94:0/1996265040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b12de8f80 0x562b12e333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f2566a00640 0 client.25730 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 >> 172.21.15.94:0/1996265040 conn(0x562b12e17bc0 msgr2=0x562b12e21ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 shutdown_connections 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 1 -- 172.21.15.94:0/1996265040 wait complete. 2024-06-03T01:15:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.522+0000 7f258a5ef740 20 client.25730 trim_cache size 0 max 16384 2024-06-03T01:15:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:15:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:15:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f25680a2da0 msgr2=0x7f25680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f25680a2da0 0x7f25680a5260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2578009fd0 tx=0x7f2578009500 comp rx=0 tx=0).stop 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2580108ee0 msgr2=0x7f258019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2580108ee0 0x7f258019a9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f256c00e9c0 tx=0x7f256c00ee80 comp rx=0 tx=0).stop 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 shutdown_connections 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f25680a2da0 0x7f25680a5260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f258019ed20 0x7f258019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2580108ee0 0x7f258019a9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 --2- 172.21.15.94:0/794608290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2580103e10 0x7f258019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 >> 172.21.15.94:0/794608290 conn(0x7f25800ff6d0 msgr2=0x7f25801019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 shutdown_connections 2024-06-03T01:15:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:41.523+0000 7f2587800640 1 -- 172.21.15.94:0/794608290 wait complete. 2024-06-03T01:15:42.083 INFO:tasks.workunit.client.0.smithi094.stdout: 1 112467 1.84 MB/sec execute 109 sec latency 19.177 ms 2024-06-03T01:15:42.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:41 smithi107 ceph-mon[36622]: pgmap v163: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-06-03T01:15:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:41 smithi094 ceph-mon[32119]: pgmap v163: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-06-03T01:15:42.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:41 smithi179 ceph-mon[37041]: pgmap v163: 129 pgs: 129 active+clean; 52 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 1.2 MiB/s wr, 106 op/s 2024-06-03T01:15:43.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 113764 1.92 MB/sec execute 110 sec latency 17.900 ms 2024-06-03T01:15:44.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 114351 1.91 MB/sec execute 111 sec latency 19.566 ms 2024-06-03T01:15:44.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:43 smithi107 ceph-mon[36622]: pgmap v164: 129 pgs: 129 active+clean; 49 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-06-03T01:15:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:43 smithi094 ceph-mon[32119]: pgmap v164: 129 pgs: 129 active+clean; 49 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-06-03T01:15:44.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:43 smithi179 ceph-mon[37041]: pgmap v164: 129 pgs: 129 active+clean; 49 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-06-03T01:15:45.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 115985 1.96 MB/sec execute 112 sec latency 20.158 ms 2024-06-03T01:15:46.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 117220 2.03 MB/sec execute 113 sec latency 18.705 ms 2024-06-03T01:15:46.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:45 smithi107 ceph-mon[36622]: pgmap v165: 129 pgs: 129 active+clean; 49 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 228 KiB/s wr, 56 op/s 2024-06-03T01:15:46.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:45 smithi094 ceph-mon[32119]: pgmap v165: 129 pgs: 129 active+clean; 49 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 228 KiB/s wr, 56 op/s 2024-06-03T01:15:46.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:45 smithi179 ceph-mon[37041]: pgmap v165: 129 pgs: 129 active+clean; 49 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 228 KiB/s wr, 56 op/s 2024-06-03T01:15:47.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 117811 2.03 MB/sec execute 114 sec latency 14.856 ms 2024-06-03T01:15:48.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 119382 2.07 MB/sec execute 115 sec latency 16.483 ms 2024-06-03T01:15:48.282 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:47 smithi107 ceph-mon[36622]: pgmap v166: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 617 KiB/s wr, 120 op/s 2024-06-03T01:15:48.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:47 smithi094 ceph-mon[32119]: pgmap v166: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 617 KiB/s wr, 120 op/s 2024-06-03T01:15:48.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:47 smithi179 ceph-mon[37041]: pgmap v166: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 617 KiB/s wr, 120 op/s 2024-06-03T01:15:49.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 120485 2.12 MB/sec execute 116 sec latency 19.900 ms 2024-06-03T01:15:50.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 121161 2.14 MB/sec execute 117 sec latency 24.567 ms 2024-06-03T01:15:50.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:49 smithi107 ceph-mon[36622]: pgmap v167: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 492 KiB/s wr, 92 op/s 2024-06-03T01:15:50.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:49 smithi094 ceph-mon[32119]: pgmap v167: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 492 KiB/s wr, 92 op/s 2024-06-03T01:15:50.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:49 smithi179 ceph-mon[37041]: pgmap v167: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 492 KiB/s wr, 92 op/s 2024-06-03T01:15:51.084 INFO:tasks.workunit.client.0.smithi094.stdout: 1 121389 2.12 MB/sec execute 118 sec latency 721.647 ms 2024-06-03T01:15:52.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 123034 2.16 MB/sec execute 119 sec latency 16.558 ms 2024-06-03T01:15:52.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:51 smithi107 ceph-mon[36622]: pgmap v168: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 92 op/s 2024-06-03T01:15:52.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:51 smithi094 ceph-mon[32119]: pgmap v168: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 92 op/s 2024-06-03T01:15:52.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:51 smithi179 ceph-mon[37041]: pgmap v168: 129 pgs: 129 active+clean; 53 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 492 KiB/s wr, 92 op/s 2024-06-03T01:15:53.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124285 2.23 MB/sec execute 120 sec latency 15.436 ms 2024-06-03T01:15:54.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124784 2.23 MB/sec execute 121 sec latency 23.573 ms 2024-06-03T01:15:54.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:53 smithi107 ceph-mon[36622]: pgmap v169: 129 pgs: 129 active+clean; 56 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 862 KiB/s wr, 151 op/s 2024-06-03T01:15:54.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:53 smithi094 ceph-mon[32119]: pgmap v169: 129 pgs: 129 active+clean; 56 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 862 KiB/s wr, 151 op/s 2024-06-03T01:15:54.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:53 smithi179 ceph-mon[37041]: pgmap v169: 129 pgs: 129 active+clean; 56 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 862 KiB/s wr, 151 op/s 2024-06-03T01:15:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 -- 172.21.15.94:0/1101774380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 msgr2=0x7fe0a8110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 --2- 172.21.15.94:0/1101774380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 0x7fe0a8110680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009ca0 tx=0x7fe0a002a940 comp rx=0 tx=0).stop 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 -- 172.21.15.94:0/1101774380 shutdown_connections 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 --2- 172.21.15.94:0/1101774380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 0x7fe0a8110680 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 --2- 172.21.15.94:0/1101774380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0a80745c0 0x7fe0a810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 --2- 172.21.15.94:0/1101774380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 0x7fe0a8074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 -- 172.21.15.94:0/1101774380 >> 172.21.15.94:0/1101774380 conn(0x7fe0a8078e30 msgr2=0x7fe0a8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 -- 172.21.15.94:0/1101774380 shutdown_connections 2024-06-03T01:15:54.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.665+0000 7fe0afa00640 1 -- 172.21.15.94:0/1101774380 wait complete. 2024-06-03T01:15:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.666+0000 7fe0afa00640 1 Processor -- start 2024-06-03T01:15:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0afa00640 1 -- start start 2024-06-03T01:15:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0afa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 0x7fe0a819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0afa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0a80745c0 0x7fe0a819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0afa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 0x7fe0a819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0afa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8114010 con 0x7fe0a8073ca0 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0afa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8113e90 con 0x7fe0a810e290 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0afa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8114190 con 0x7fe0a80745c0 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0ad000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 0x7fe0a819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.667+0000 7fe0ad000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 0x7fe0a819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47742/0 (socket says 172.21.15.94:47742) 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ad000640 1 -- 172.21.15.94:0/159932624 learned_addr learned my addr 172.21.15.94:0/159932624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ada00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 0x7fe0a819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0a80745c0 0x7fe0a819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ad000640 1 -- 172.21.15.94:0/159932624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0a80745c0 msgr2=0x7fe0a819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ad000640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0a80745c0 0x7fe0a819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ad000640 1 -- 172.21.15.94:0/159932624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 msgr2=0x7fe0a819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ad000640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 0x7fe0a819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ad000640 1 -- 172.21.15.94:0/159932624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a819f310 con 0x7fe0a8073ca0 2024-06-03T01:15:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ada00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 0x7fe0a819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:15:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.668+0000 7fe0ad000640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 0x7fe0a819a2d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00e9f0 tx=0x7fe09c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.669+0000 7fe0a5600640 1 -- 172.21.15.94:0/159932624 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c00cd80 con 0x7fe0a8073ca0 2024-06-03T01:15:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.669+0000 7fe0a5600640 1 -- 172.21.15.94:0/159932624 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe09c010910 con 0x7fe0a8073ca0 2024-06-03T01:15:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.669+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a819f600 con 0x7fe0a8073ca0 2024-06-03T01:15:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.669+0000 7fe0a5600640 1 -- 172.21.15.94:0/159932624 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c018990 con 0x7fe0a8073ca0 2024-06-03T01:15:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.669+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a81a6e40 con 0x7fe0a8073ca0 2024-06-03T01:15:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.672+0000 7fe0b277c740 1 Processor -- start 2024-06-03T01:15:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.672+0000 7fe0b277c740 1 -- start start 2024-06-03T01:15:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd6040 0x5570a2bd64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0b277c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570a2b3a420 con 0x5570a2bd6040 2024-06-03T01:15:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0b277c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5570a2ae1c50 con 0x5570a2bd75b0 2024-06-03T01:15:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0b277c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5570a2b363e0 con 0x5570a2be0dc0 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0a5600640 1 -- 172.21.15.94:0/159932624 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe09c010ac0 con 0x7fe0a8073ca0 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0ada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0ada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bd5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:46530/0 (socket says 172.21.15.94:46530) 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0ada00640 1 -- 172.21.15.94:0/3292857377 learned_addr learned my addr 172.21.15.94:0/3292857377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.673+0000 7fe0ad000640 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd6040 0x5570a2bd64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe0a7e00640 1 -- 172.21.15.94:0/3292857377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 msgr2=0x5570a2bd5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bd5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe0a7e00640 1 -- 172.21.15.94:0/3292857377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd6040 msgr2=0x5570a2bd64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd6040 0x5570a2bd64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe0a7e00640 1 -- 172.21.15.94:0/3292857377 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a284e210 con 0x5570a2be0dc0 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be31b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe09800bb70 tx=0x7fe0980107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7caad97876ad4e74 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe096000640 1 -- 172.21.15.94:0/3292857377 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098004030 con 0x5570a2be0dc0 2024-06-03T01:15:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe096000640 1 -- 172.21.15.94:0/3292857377 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0980041d0 con 0x5570a2be0dc0 2024-06-03T01:15:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.674+0000 7fe096000640 1 -- 172.21.15.94:0/3292857377 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098011680 con 0x5570a2be0dc0 2024-06-03T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0a5600640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0900a2da0 0x7fe0900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0a5600640 1 -- 172.21.15.94:0/159932624 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe09c0cfb00 con 0x7fe0a8073ca0 2024-06-03T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 -- 172.21.15.94:0/3292857377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 msgr2=0x5570a2be31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be31b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe09800bb70 tx=0x7fe0980107a0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0ada00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0900a2da0 0x7fe0900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 -- 172.21.15.94:0/3292857377 shutdown_connections 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be31b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd6040 0x5570a2bd64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 --2- 172.21.15.94:0/3292857377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bd5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 -- 172.21.15.94:0/3292857377 >> 172.21.15.94:0/3292857377 conn(0x5570a2b9ef80 msgr2=0x5570a2bd2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 -- 172.21.15.94:0/3292857377 shutdown_connections 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.675+0000 7fe0b277c740 1 -- 172.21.15.94:0/3292857377 wait complete. 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0ada00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0900a2da0 0x7fe0900a5260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009a60 tx=0x7fe0a0005d80 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 -- start start 2024-06-03T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 Processor -- start 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bdde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bde890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5570a2be3b90 con 0x5570a2be0dc0 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5570a2af50b0 con 0x5570a2bd75b0 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.676+0000 7fe0b277c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5570a2be3d10 con 0x5570a2bd6040 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0b277c740 10 client.0 _ll_register_callbacks cb 0x5570a2ae0f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bde890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bde890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47764/0 (socket says 172.21.15.94:47764) 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 -- 172.21.15.94:0/2626615445 learned_addr learned my addr 172.21.15.94:0/2626615445 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bde350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ad000640 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bdde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 -- 172.21.15.94:0/2626615445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 msgr2=0x5570a2bdde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bdde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 -- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 msgr2=0x5570a2bde350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bde350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 -- 172.21.15.94:0/2626615445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a291a620 con 0x5570a2be0dc0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe0ada00640 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bde890 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0a002ae60 tx=0x7fe0a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e5f9bb19483b83 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.677+0000 7fe08aa00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.678+0000 7fe08aa00640 1 -- 172.21.15.94:0/2626615445 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0056060 con 0x5570a2be0dc0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.678+0000 7fe08aa00640 1 -- 172.21.15.94:0/2626615445 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe0a0030d60 con 0x5570a2be0dc0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.678+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0980041d0 con 0x5570a2be0dc0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.678+0000 7fe08aa00640 1 -- 172.21.15.94:0/2626615445 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe0a005d050 con 0x5570a2be0dc0 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.678+0000 7fe0b277c740 10 client.15618 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.678+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5570a2beff60 con 0x5570a2be0dc0 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.678+0000 7fe08aa00640 1 -- 172.21.15.94:0/2626615445 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0980041d0 con 0x5570a2be0dc0 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe08aa00640 1 -- 172.21.15.94:0/2626615445 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7fe0a002f070 con 0x5570a2be0dc0 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe0b277c740 10 client.15618 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe0b277c740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe0b277c740 20 client.15618 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2bf1420 0x5570a2bf3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe0b277c740 4 client.15618 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570a2b47b50 con 0x5570a2bf1420 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.679+0000 7fe0ad000640 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2bf1420 0x5570a2bf3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.680+0000 7fe0ad000640 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2bf1420 0x5570a2bf3810 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2321e5d1006708c0 server_cookie=fae95cb9f1ebcf46 in_seq=0 out_seq=0 2024-06-03T01:15:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.680+0000 7fe08aa00640 10 client.15618 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.680+0000 7fe08aa00640 1 -- 172.21.15.94:0/2626615445 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5570a2b47b50 con 0x5570a2bf1420 2024-06-03T01:15:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.680+0000 7fe08aa00640 10 client.15618 handle_command_reply: tid=0 2024-06-03T01:15:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.680+0000 7fe0b277c740 1 client.15618 shutdown 2024-06-03T01:15:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 msgr2=0x5570a2bde890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bde890 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe0a002ae60 tx=0x7fe0a00095c0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 shutdown_connections 2024-06-03T01:15:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bde890 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2bf1420 0x5570a2bf3810 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bde350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2626615445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bdde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe08aa00640 0 client.15618 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 >> 172.21.15.94:0/2626615445 conn(0x5570a2b9ef80 msgr2=0x5570a2bd84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 shutdown_connections 2024-06-03T01:15:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 1 -- 172.21.15.94:0/2626615445 wait complete. 2024-06-03T01:15:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.681+0000 7fe0b277c740 20 client.15618 trim_cache size 0 max 16384 2024-06-03T01:15:54.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 Processor -- start 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 -- start start 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bf90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bf9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bd0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81b0ee0 con 0x5570a2be0dc0 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81b2a50 con 0x5570a2bd75b0 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.697+0000 7fe0b277c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81b15b0 con 0x5570a2bd6040 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bd0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bd0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47766/0 (socket says 172.21.15.94:47766) 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 -- 172.21.15.94:0/2866589187 learned_addr learned my addr 172.21.15.94:0/2866589187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:54.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ad000640 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bf90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bf9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 -- 172.21.15.94:0/2866589187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 msgr2=0x5570a2bf90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bf90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 -- 172.21.15.94:0/2866589187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 msgr2=0x5570a2bf9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bf9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 -- 172.21.15.94:0/2866589187 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a2bd6b10 con 0x5570a2be0dc0 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bf9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ad000640 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bf90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.698+0000 7fe0ada00640 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bd0f80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe0a003a780 tx=0x7fe0a002fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff577f1f9573d0a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.699+0000 7fe095600640 1 -- 172.21.15.94:0/2866589187 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0056040 con 0x5570a2be0dc0 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.699+0000 7fe095600640 1 -- 172.21.15.94:0/2866589187 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a003a430 con 0x5570a2be0dc0 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.699+0000 7fe095600640 1 -- 172.21.15.94:0/2866589187 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0034040 con 0x5570a2be0dc0 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.699+0000 7fe0b277c740 1 -- 172.21.15.94:0/2866589187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 msgr2=0x5570a2bd0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.699+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bd0f80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe0a003a780 tx=0x7fe0a002fd00 comp rx=0 tx=0).stop 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 -- 172.21.15.94:0/2866589187 shutdown_connections 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2be0dc0 0x5570a2bd0f80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd75b0 0x5570a2bf9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 --2- 172.21.15.94:0/2866589187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2bd6040 0x5570a2bf90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 -- 172.21.15.94:0/2866589187 >> 172.21.15.94:0/2866589187 conn(0x5570a2b9ef80 msgr2=0x5570a2bd90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 -- 172.21.15.94:0/2866589187 shutdown_connections 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 -- 172.21.15.94:0/2866589187 wait complete. 2024-06-03T01:15:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 -- start start 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.700+0000 7fe0b277c740 1 Processor -- start 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd6040 0x5570a2be9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd75b0 0x5570a2be98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0b277c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0b277c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81b2380 con 0x5570a2bd75b0 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0b277c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81b1c80 con 0x5570a2bd6040 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0b277c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81aeec0 con 0x5570a2be0dc0 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0b277c740 10 client.0 _ll_register_callbacks cb 0x7fe0a0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:15:54.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd6040 0x5570a2be9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd75b0 0x5570a2be98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd6040 0x5570a2be9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:46554/0 (socket says 172.21.15.94:46554) 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 -- 172.21.15.94:0/4245827262 learned_addr learned my addr 172.21.15.94:0/4245827262 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ada00640 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 -- 172.21.15.94:0/4245827262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 msgr2=0x5570a2be4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 -- 172.21.15.94:0/4245827262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd75b0 msgr2=0x5570a2be98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd75b0 0x5570a2be98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.701+0000 7fe0ad000640 1 -- 172.21.15.94:0/4245827262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a2be50d0 con 0x5570a2bd6040 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0ad000640 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd6040 0x5570a2be9390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe09c098170 tx=0x7fe09c0d35b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cdabf1ee745525 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe08aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe08aa00640 1 -- 172.21.15.94:0/4245827262 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0d7040 con 0x5570a2bd6040 2024-06-03T01:15:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe08aa00640 1 -- 172.21.15.94:0/4245827262 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe09c098380 con 0x5570a2bd6040 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a003a430 con 0x5570a2bd6040 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe08aa00640 1 -- 172.21.15.94:0/4245827262 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe09c09bd10 con 0x5570a2bd6040 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0b277c740 10 client.25748 fetch_fsmap learned FSMap version 36 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5570a2c5bd20 con 0x5570a2bd6040 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe08aa00640 1 -- 172.21.15.94:0/4245827262 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a003a430 con 0x5570a2bd6040 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe08aa00640 1 -- 172.21.15.94:0/4245827262 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7fe09c018b30 con 0x5570a2bd6040 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0b277c740 10 client.25748 fetch_fsmap finished waiting for FSMap version 36 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0b277c740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0b277c740 20 client.25748 populate_metadata read hostname 'smithi094' 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.702+0000 7fe0b277c740 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2c5c420 0x5570a2c5e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.703+0000 7fe0b277c740 4 client.25748 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:15:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.703+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570a2b41ea0 con 0x5570a2c5c420 2024-06-03T01:15:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.703+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2c5c420 0x5570a2c5e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:15:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.703+0000 7fe0a7e00640 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2c5c420 0x5570a2c5e840 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f39240f441fb74 server_cookie=2022ec12a8e6591c in_seq=0 out_seq=0 2024-06-03T01:15:54.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.703+0000 7fe08aa00640 10 client.25748 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe08aa00640 1 -- 172.21.15.94:0/4245827262 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5570a2b41ea0 con 0x5570a2c5c420 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe08aa00640 10 client.25748 handle_command_reply: tid=0 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 client.25748 shutdown 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd6040 msgr2=0x5570a2be9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd6040 0x5570a2be9390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe09c098170 tx=0x7fe09c0d35b0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 shutdown_connections 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5570a2be0dc0 0x5570a2be4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5570a2bd75b0 0x5570a2be98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5570a2c5c420 0x5570a2c5e840 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 --2- 172.21.15.94:0/4245827262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570a2bd6040 0x5570a2be9390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe08aa00640 0 client.25748 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 >> 172.21.15.94:0/4245827262 conn(0x5570a2b9ef80 msgr2=0x5570a2bdddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.760+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 shutdown_connections 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.761+0000 7fe0b277c740 1 -- 172.21.15.94:0/4245827262 wait complete. 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.761+0000 7fe0b277c740 20 client.25748 trim_cache size 0 max 16384 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:15:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:15:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.761+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0900a2da0 msgr2=0x7fe0900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.761+0000 7fe0afa00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0900a2da0 0x7fe0900a5260 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009a60 tx=0x7fe0a0005d80 comp rx=0 tx=0).stop 2024-06-03T01:15:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.761+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 msgr2=0x7fe0a819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:15:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.761+0000 7fe0afa00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 0x7fe0a819a2d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00e9f0 tx=0x7fe09c00eeb0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 shutdown_connections 2024-06-03T01:15:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe0900a2da0 0x7fe0900a5260 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810e290 0x7fe0a819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe0a80745c0 0x7fe0a819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 --2- 172.21.15.94:0/159932624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe0a8073ca0 0x7fe0a819a2d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:15:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 >> 172.21.15.94:0/159932624 conn(0x7fe0a8078e30 msgr2=0x7fe0a80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:15:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 shutdown_connections 2024-06-03T01:15:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:15:54.762+0000 7fe0afa00640 1 -- 172.21.15.94:0/159932624 wait complete. 2024-06-03T01:15:55.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124849 2.21 MB/sec execute 122 sec latency 956.432 ms 2024-06-03T01:15:56.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124868 2.19 MB/sec execute 123 sec latency 1924.752 ms 2024-06-03T01:15:56.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:55 smithi107 ceph-mon[36622]: pgmap v170: 129 pgs: 129 active+clean; 56 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 760 KiB/s wr, 122 op/s 2024-06-03T01:15:56.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:55 smithi094 ceph-mon[32119]: pgmap v170: 129 pgs: 129 active+clean; 56 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 760 KiB/s wr, 122 op/s 2024-06-03T01:15:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:55 smithi179 ceph-mon[37041]: pgmap v170: 129 pgs: 129 active+clean; 56 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 760 KiB/s wr, 122 op/s 2024-06-03T01:15:57.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124973 2.18 MB/sec execute 124 sec latency 853.240 ms 2024-06-03T01:15:58.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124973 2.16 MB/sec execute 125 sec latency 1853.386 ms 2024-06-03T01:15:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:57 smithi107 ceph-mon[36622]: pgmap v171: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 975 KiB/s wr, 181 op/s 2024-06-03T01:15:58.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:57 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:15:58.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:57 smithi094 ceph-mon[32119]: pgmap v171: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 975 KiB/s wr, 181 op/s 2024-06-03T01:15:58.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:57 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:15:58.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:57 smithi179 ceph-mon[37041]: pgmap v171: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 975 KiB/s wr, 181 op/s 2024-06-03T01:15:58.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:57 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:15:59.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124973 2.14 MB/sec execute 126 sec latency 2853.557 ms 2024-06-03T01:16:00.085 INFO:tasks.workunit.client.0.smithi094.stdout: 1 124973 2.12 MB/sec execute 127 sec latency 3853.648 ms 2024-06-03T01:16:00.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:15:59 smithi107 ceph-mon[36622]: pgmap v172: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 585 KiB/s wr, 117 op/s 2024-06-03T01:16:00.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:15:59 smithi094 ceph-mon[32119]: pgmap v172: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 585 KiB/s wr, 117 op/s 2024-06-03T01:16:00.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:15:59 smithi179 ceph-mon[37041]: pgmap v172: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 585 KiB/s wr, 117 op/s 2024-06-03T01:16:01.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 125198 2.11 MB/sec execute 128 sec latency 4351.914 ms 2024-06-03T01:16:02.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 126271 2.13 MB/sec execute 129 sec latency 87.133 ms 2024-06-03T01:16:02.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:01 smithi107 ceph-mon[36622]: pgmap v173: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 585 KiB/s wr, 117 op/s 2024-06-03T01:16:02.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:01 smithi094 ceph-mon[32119]: pgmap v173: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 585 KiB/s wr, 117 op/s 2024-06-03T01:16:02.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:01 smithi179 ceph-mon[37041]: pgmap v173: 129 pgs: 129 active+clean; 54 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 585 KiB/s wr, 117 op/s 2024-06-03T01:16:03.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 127644 2.20 MB/sec execute 130 sec latency 16.627 ms 2024-06-03T01:16:04.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128211 2.20 MB/sec execute 131 sec latency 19.576 ms 2024-06-03T01:16:04.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:03 smithi107 ceph-mon[36622]: pgmap v174: 129 pgs: 129 active+clean; 55 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 879 KiB/s wr, 165 op/s 2024-06-03T01:16:04.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:03 smithi094 ceph-mon[32119]: pgmap v174: 129 pgs: 129 active+clean; 55 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 879 KiB/s wr, 165 op/s 2024-06-03T01:16:04.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:03 smithi179 ceph-mon[37041]: pgmap v174: 129 pgs: 129 active+clean; 55 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 879 KiB/s wr, 165 op/s 2024-06-03T01:16:05.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128449 2.19 MB/sec execute 132 sec latency 715.111 ms 2024-06-03T01:16:06.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128466 2.17 MB/sec execute 133 sec latency 1681.965 ms 2024-06-03T01:16:06.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:05 smithi107 ceph-mon[36622]: pgmap v175: 129 pgs: 129 active+clean; 55 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 509 KiB/s wr, 106 op/s 2024-06-03T01:16:06.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:05 smithi094 ceph-mon[32119]: pgmap v175: 129 pgs: 129 active+clean; 55 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 509 KiB/s wr, 106 op/s 2024-06-03T01:16:06.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:05 smithi179 ceph-mon[37041]: pgmap v175: 129 pgs: 129 active+clean; 55 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 509 KiB/s wr, 106 op/s 2024-06-03T01:16:07.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128518 2.16 MB/sec execute 134 sec latency 934.200 ms 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 -- 172.21.15.94:0/1872165026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 msgr2=0x7fe2bc107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 --2- 172.21.15.94:0/1872165026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc107e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009960 tx=0x7fe2b002a460 comp rx=0 tx=0).stop 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 -- 172.21.15.94:0/1872165026 shutdown_connections 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 --2- 172.21.15.94:0/1872165026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2bc06acd0 0x7fe2bc10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 --2- 172.21.15.94:0/1872165026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 0x7fe2bc06a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 --2- 172.21.15.94:0/1872165026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc107e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 -- 172.21.15.94:0/1872165026 >> 172.21.15.94:0/1872165026 conn(0x7fe2bc076be0 msgr2=0x7fe2bc076ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 -- 172.21.15.94:0/1872165026 shutdown_connections 2024-06-03T01:16:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.903+0000 7fe2c3600640 1 -- 172.21.15.94:0/1872165026 wait complete. 2024-06-03T01:16:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.904+0000 7fe2c3600640 1 Processor -- start 2024-06-03T01:16:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.904+0000 7fe2c3600640 1 -- start start 2024-06-03T01:16:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.904+0000 7fe2c3600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 0x7fe2bc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.904+0000 7fe2c3600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2bc06acd0 0x7fe2bc198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.904+0000 7fe2c3600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.904+0000 7fe2c3600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc111dc0 con 0x7fe2bc06a310 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.904+0000 7fe2c3600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc111c40 con 0x7fe2bc107a50 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c3600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc111f40 con 0x7fe2bc06acd0 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 0x7fe2bc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 0x7fe2bc198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56146/0 (socket says 172.21.15.94:56146) 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 -- 172.21.15.94:0/347445922 learned_addr learned my addr 172.21.15.94:0/347445922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:49350/0 (socket says 172.21.15.94:49350) 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2bc06acd0 0x7fe2bc198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 -- 172.21.15.94:0/347445922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2bc06acd0 msgr2=0x7fe2bc198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2bc06acd0 0x7fe2bc198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 -- 172.21.15.94:0/347445922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 msgr2=0x7fe2bc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c0c00640 1 -- 172.21.15.94:0/347445922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc19d230 con 0x7fe2bc06a310 2024-06-03T01:16:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.905+0000 7fe2c1600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:16:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.906+0000 7fe2c0c00640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 0x7fe2bc198240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.906+0000 7fe2b9600640 1 -- 172.21.15.94:0/347445922 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b003f070 con 0x7fe2bc06a310 2024-06-03T01:16:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.906+0000 7fe2b9600640 1 -- 172.21.15.94:0/347445922 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0031bc0 con 0x7fe2bc06a310 2024-06-03T01:16:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.906+0000 7fe2b9600640 1 -- 172.21.15.94:0/347445922 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0035030 con 0x7fe2bc06a310 2024-06-03T01:16:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.906+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc19d4c0 con 0x7fe2bc06a310 2024-06-03T01:16:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.906+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1a4db0 con 0x7fe2bc06a310 2024-06-03T01:16:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.909+0000 7fe2c642e740 1 Processor -- start 2024-06-03T01:16:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.909+0000 7fe2b9600640 1 -- 172.21.15.94:0/347445922 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe2b0042070 con 0x7fe2bc06a310 2024-06-03T01:16:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c642e740 1 -- start start 2024-06-03T01:16:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae80bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae80c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae8191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c642e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556bae770420 con 0x556bae816dc0 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c642e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556bae717c50 con 0x556bae80c040 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c642e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556bae76c3e0 con 0x556bae80d5b0 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae80bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae80bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49402/0 (socket says 172.21.15.94:49402) 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c1600640 1 -- 172.21.15.94:0/2589856045 learned_addr learned my addr 172.21.15.94:0/2589856045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae8191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.910+0000 7fe2c0c00640 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae80c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2c1600640 1 -- 172.21.15.94:0/2589856045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 msgr2=0x556bae80c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae80c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2c1600640 1 -- 172.21.15.94:0/2589856045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 msgr2=0x556bae8191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae8191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2c1600640 1 -- 172.21.15.94:0/2589856045 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bae484210 con 0x556bae80d5b0 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae8191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2b9600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe2a40a2e20 0x7fe2a40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae80bb00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40117c0 tx=0x7fe2b4011c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c442f6b6ffb0c67 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2b9600640 1 -- 172.21.15.94:0/347445922 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe2b00efcb0 con 0x7fe2bc06a310 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2c0c00640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe2a40a2e20 0x7fe2a40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2adc00640 1 -- 172.21.15.94:0/2589856045 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4003c30 con 0x556bae80d5b0 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2adc00640 1 -- 172.21.15.94:0/2589856045 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b400fe80 con 0x556bae80d5b0 2024-06-03T01:16:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.911+0000 7fe2adc00640 1 -- 172.21.15.94:0/2589856045 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40193e0 con 0x556bae80d5b0 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 -- 172.21.15.94:0/2589856045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 msgr2=0x556bae80bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae80bb00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40117c0 tx=0x7fe2b4011c80 comp rx=0 tx=0).stop 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c0c00640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe2a40a2e20 0x7fe2a40a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x556bae80ed10 tx=0x7fe2b00f3760 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 -- 172.21.15.94:0/2589856045 shutdown_connections 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae8191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae80c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2589856045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae80bb00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 -- 172.21.15.94:0/2589856045 >> 172.21.15.94:0/2589856045 conn(0x556bae7d4f80 msgr2=0x556bae808630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 -- 172.21.15.94:0/2589856045 shutdown_connections 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.912+0000 7fe2c642e740 1 -- 172.21.15.94:0/2589856045 wait complete. 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 -- start start 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 Processor -- start 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae813da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae814820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556bae819b90 con 0x556bae80d5b0 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556bae72b0b0 con 0x556bae816dc0 2024-06-03T01:16:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.913+0000 7fe2c642e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556bae819d10 con 0x556bae80c040 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2c642e740 10 client.0 _ll_register_callbacks cb 0x556bae716f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8142e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56172/0 (socket says 172.21.15.94:56172) 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 -- 172.21.15.94:0/2263798897 learned_addr learned my addr 172.21.15.94:0/2263798897 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae813da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae814820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 -- 172.21.15.94:0/2263798897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 msgr2=0x556bae813da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae813da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 -- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 msgr2=0x556bae814820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae814820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2bbe00640 1 -- 172.21.15.94:0/2263798897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bae550620 con 0x556bae80d5b0 2024-06-03T01:16:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.914+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae814820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8142e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8012700 tx=0x7fe2a8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2360ba66babb22 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2a2000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2a2000640 1 -- 172.21.15.94:0/2263798897 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800f920 con 0x556bae80d5b0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556bae77db50 con 0x556bae80d5b0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2a2000640 1 -- 172.21.15.94:0/2263798897 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe2a800fac0 con 0x556bae80d5b0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2a2000640 1 -- 172.21.15.94:0/2263798897 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe2a800fcc0 con 0x556bae80d5b0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2c642e740 10 client.15636 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556bae82a360 con 0x556bae80d5b0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.915+0000 7fe2a2000640 1 -- 172.21.15.94:0/2263798897 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556bae77db50 con 0x556bae80d5b0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2a2000640 1 -- 172.21.15.94:0/2263798897 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe2a801c420 con 0x556bae80d5b0 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2c642e740 10 client.15636 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2c642e740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2c642e740 20 client.15636 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae82b8c0 0x556bae82dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2c642e740 4 client.15636 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:16:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bae777ea0 con 0x556bae82b8c0 2024-06-03T01:16:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.916+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae82b8c0 0x556bae82dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.917+0000 7fe2c1600640 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae82b8c0 0x556bae82dcb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2690d9dbc55af6ce server_cookie=37585b40b63d5fdc in_seq=0 out_seq=0 2024-06-03T01:16:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.917+0000 7fe2a2000640 10 client.15636 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2a2000640 1 -- 172.21.15.94:0/2263798897 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556bae777ea0 con 0x556bae82b8c0 2024-06-03T01:16:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2a2000640 10 client.15636 handle_command_reply: tid=0 2024-06-03T01:16:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 client.15636 shutdown 2024-06-03T01:16:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 msgr2=0x556bae8142e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8142e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8012700 tx=0x7fe2a8012bc0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 shutdown_connections 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae82b8c0 0x556bae82dcb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae814820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8142e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 --2- 172.21.15.94:0/2263798897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae813da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2a2000640 0 client.15636 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 >> 172.21.15.94:0/2263798897 conn(0x556bae7d4f80 msgr2=0x556bae80e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 shutdown_connections 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 1 -- 172.21.15.94:0/2263798897 wait complete. 2024-06-03T01:16:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.918+0000 7fe2c642e740 20 client.15636 trim_cache size 0 max 16384 2024-06-03T01:16:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 Processor -- start 2024-06-03T01:16:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 -- start start 2024-06-03T01:16:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae803750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae814640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae807f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1acba0 con 0x556bae816dc0 2024-06-03T01:16:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ae6e0 con 0x556bae80c040 2024-06-03T01:16:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c642e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ad270 con 0x556bae80d5b0 2024-06-03T01:16:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae807f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.935+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae803750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae803750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:49368/0 (socket says 172.21.15.94:49368) 2024-06-03T01:16:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 -- 172.21.15.94:0/1424229601 learned_addr learned my addr 172.21.15.94:0/1424229601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae814640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 -- 172.21.15.94:0/1424229601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 msgr2=0x556bae814640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae814640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 -- 172.21.15.94:0/1424229601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 msgr2=0x556bae807f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae807f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 -- 172.21.15.94:0/1424229601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bae812d50 con 0x556bae80c040 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2c1600640 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae803750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005c40 tx=0x7fe2b4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6399892edb962159 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2ad200640 1 -- 172.21.15.94:0/1424229601 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004490 con 0x556bae80c040 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2ad200640 1 -- 172.21.15.94:0/1424229601 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4011050 con 0x556bae80c040 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.936+0000 7fe2ad200640 1 -- 172.21.15.94:0/1424229601 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40048b0 con 0x556bae80c040 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 -- 172.21.15.94:0/1424229601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 msgr2=0x556bae803750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae803750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005c40 tx=0x7fe2b4005fd0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 -- 172.21.15.94:0/1424229601 shutdown_connections 2024-06-03T01:16:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae816dc0 0x556bae807f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80d5b0 0x556bae814640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 --2- 172.21.15.94:0/1424229601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae80c040 0x556bae803750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 -- 172.21.15.94:0/1424229601 >> 172.21.15.94:0/1424229601 conn(0x556bae7d4f80 msgr2=0x556bae80e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 -- 172.21.15.94:0/1424229601 shutdown_connections 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.937+0000 7fe2c642e740 1 -- 172.21.15.94:0/1424229601 wait complete. 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 -- start start 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 Processor -- start 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae823460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae8100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ae010 con 0x556bae80d5b0 2024-06-03T01:16:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ad940 con 0x556bae816dc0 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1aab80 con 0x556bae80c040 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c642e740 10 client.0 _ll_register_callbacks cb 0x7fe2a8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.938+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae823460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae823460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:49412/0 (socket says 172.21.15.94:49412) 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8239a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56196/0 (socket says 172.21.15.94:56196) 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2c1600640 1 -- 172.21.15.94:0/4219449120 learned_addr learned my addr 172.21.15.94:0/4219449120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2c1600640 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae8100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2bbe00640 1 -- 172.21.15.94:0/4219449120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 msgr2=0x556bae823460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae823460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2bbe00640 1 -- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 msgr2=0x556bae8100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:07.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae8100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2bbe00640 1 -- 172.21.15.94:0/4219449120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bae824060 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2bbe00640 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8239a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8001eb0 tx=0x7fe2a800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b984f9c4aebe43 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2a2000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2a2000640 1 -- 172.21.15.94:0/4219449120 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a801cce0 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.939+0000 7fe2a2000640 1 -- 172.21.15.94:0/4219449120 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe2a801c420 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b4011050 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2a2000640 1 -- 172.21.15.94:0/4219449120 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe2a801c660 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 10 client.15642 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556bae895b60 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2a2000640 1 -- 172.21.15.94:0/4219449120 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4011050 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2a2000640 1 -- 172.21.15.94:0/4219449120 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe2a8003b40 con 0x556bae80d5b0 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 10 client.15642 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 20 client.15642 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae8962e0 0x556bae898700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 4 client.15642 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:16:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.940+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bae6f57a0 con 0x556bae8962e0 2024-06-03T01:16:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.941+0000 7fe2c1600640 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae8962e0 0x556bae898700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.941+0000 7fe2c1600640 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae8962e0 0x556bae898700 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48a4939f375a5bd server_cookie=6d35626f62cf3aff in_seq=0 out_seq=0 2024-06-03T01:16:07.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.941+0000 7fe2a2000640 10 client.15642 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.975+0000 7fe2a2000640 1 -- 172.21.15.94:0/4219449120 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556bae6f57a0 con 0x556bae8962e0 2024-06-03T01:16:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.975+0000 7fe2a2000640 10 client.15642 handle_command_reply: tid=0 2024-06-03T01:16:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.975+0000 7fe2c642e740 1 client.15642 shutdown 2024-06-03T01:16:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.975+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 msgr2=0x556bae8239a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.975+0000 7fe2c642e740 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8239a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8001eb0 tx=0x7fe2a800f820 comp rx=0 tx=0).stop 2024-06-03T01:16:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 shutdown_connections 2024-06-03T01:16:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556bae8962e0 0x556bae898700 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bae816dc0 0x556bae8100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bae80d5b0 0x556bae8239a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 --2- 172.21.15.94:0/4219449120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556bae80c040 0x556bae823460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2a2000640 0 client.15642 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 >> 172.21.15.94:0/4219449120 conn(0x556bae7d4f80 msgr2=0x556bae8262e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 shutdown_connections 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 1 -- 172.21.15.94:0/4219449120 wait complete. 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.976+0000 7fe2c642e740 20 client.15642 trim_cache size 0 max 16384 2024-06-03T01:16:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:16:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:16:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:16:07.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe2a40a2e20 msgr2=0x7fe2a40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe2a40a2e20 0x7fe2a40a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x556bae80ed10 tx=0x7fe2b00f3760 comp rx=0 tx=0).stop 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 msgr2=0x7fe2bc198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 0x7fe2bc198240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b0003c10 comp rx=0 tx=0).stop 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 shutdown_connections 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe2a40a2e20 0x7fe2a40a52e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2bc107a50 0x7fe2bc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2bc06acd0 0x7fe2bc198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 --2- 172.21.15.94:0/347445922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2bc06a310 0x7fe2bc198240 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 >> 172.21.15.94:0/347445922 conn(0x7fe2bc076be0 msgr2=0x7fe2bc0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 shutdown_connections 2024-06-03T01:16:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:07.977+0000 7fe2c3600640 1 -- 172.21.15.94:0/347445922 wait complete. 2024-06-03T01:16:08.086 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128518 2.14 MB/sec execute 135 sec latency 1934.334 ms 2024-06-03T01:16:08.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:07 smithi107 ceph-mon[36622]: pgmap v176: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 902 KiB/s wr, 150 op/s 2024-06-03T01:16:08.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:07 smithi094 ceph-mon[32119]: pgmap v176: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 902 KiB/s wr, 150 op/s 2024-06-03T01:16:08.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:07 smithi179 ceph-mon[37041]: pgmap v176: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 902 KiB/s wr, 150 op/s 2024-06-03T01:16:09.087 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128518 2.12 MB/sec execute 136 sec latency 2934.538 ms 2024-06-03T01:16:10.087 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128518 2.11 MB/sec execute 137 sec latency 3934.684 ms 2024-06-03T01:16:10.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:10 smithi107 ceph-mon[36622]: pgmap v177: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 687 KiB/s wr, 91 op/s 2024-06-03T01:16:10.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:10 smithi094 ceph-mon[32119]: pgmap v177: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 687 KiB/s wr, 91 op/s 2024-06-03T01:16:10.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:10 smithi179 ceph-mon[37041]: pgmap v177: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 687 KiB/s wr, 91 op/s 2024-06-03T01:16:11.087 INFO:tasks.workunit.client.0.smithi094.stdout: 1 128537 2.09 MB/sec execute 138 sec latency 4899.882 ms 2024-06-03T01:16:12.087 INFO:tasks.workunit.client.0.smithi094.stdout: 1 130157 2.13 MB/sec execute 139 sec latency 22.826 ms 2024-06-03T01:16:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:12 smithi107 ceph-mon[36622]: pgmap v178: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 687 KiB/s wr, 91 op/s 2024-06-03T01:16:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:12 smithi094 ceph-mon[32119]: pgmap v178: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 687 KiB/s wr, 91 op/s 2024-06-03T01:16:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:12 smithi179 ceph-mon[37041]: pgmap v178: 129 pgs: 129 active+clean; 59 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 687 KiB/s wr, 91 op/s 2024-06-03T01:16:13.087 INFO:tasks.workunit.client.0.smithi094.stdout: 1 131477 2.19 MB/sec execute 140 sec latency 16.052 ms 2024-06-03T01:16:14.087 INFO:tasks.workunit.client.0.smithi094.stdout: 1 131943 2.18 MB/sec execute 141 sec latency 192.947 ms 2024-06-03T01:16:14.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:14 smithi107 ceph-mon[36622]: pgmap v179: 129 pgs: 129 active+clean; 58 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 814 KiB/s wr, 122 op/s 2024-06-03T01:16:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:14 smithi094 ceph-mon[32119]: pgmap v179: 129 pgs: 129 active+clean; 58 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 814 KiB/s wr, 122 op/s 2024-06-03T01:16:14.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:14 smithi179 ceph-mon[37041]: pgmap v179: 129 pgs: 129 active+clean; 58 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 814 KiB/s wr, 122 op/s 2024-06-03T01:16:15.088 INFO:tasks.workunit.client.0.smithi094.stdout: 1 131943 2.17 MB/sec execute 142 sec latency 1193.088 ms 2024-06-03T01:16:16.088 INFO:tasks.workunit.client.0.smithi094.stdout: 1 131962 2.15 MB/sec execute 143 sec latency 2157.703 ms 2024-06-03T01:16:16.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:16 smithi107 ceph-mon[36622]: pgmap v180: 129 pgs: 129 active+clean; 58 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 520 KiB/s wr, 75 op/s 2024-06-03T01:16:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:16 smithi094 ceph-mon[32119]: pgmap v180: 129 pgs: 129 active+clean; 58 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 520 KiB/s wr, 75 op/s 2024-06-03T01:16:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:16 smithi179 ceph-mon[37041]: pgmap v180: 129 pgs: 129 active+clean; 58 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 520 KiB/s wr, 75 op/s 2024-06-03T01:16:17.088 INFO:tasks.workunit.client.0.smithi094.stdout: 1 133655 2.18 MB/sec execute 144 sec latency 31.109 ms 2024-06-03T01:16:18.088 INFO:tasks.workunit.client.0.smithi094.stdout: 1 135001 2.25 MB/sec execute 145 sec latency 18.037 ms 2024-06-03T01:16:18.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:18 smithi107 ceph-mon[36622]: pgmap v181: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 790 KiB/s wr, 122 op/s 2024-06-03T01:16:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:18 smithi094 ceph-mon[32119]: pgmap v181: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 790 KiB/s wr, 122 op/s 2024-06-03T01:16:18.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:18 smithi179 ceph-mon[37041]: pgmap v181: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 790 KiB/s wr, 122 op/s 2024-06-03T01:16:19.088 INFO:tasks.workunit.client.0.smithi094.stdout: 1 135543 2.24 MB/sec execute 146 sec latency 94.492 ms 2024-06-03T01:16:20.088 INFO:tasks.workunit.client.0.smithi094.stdout: 1 135543 2.22 MB/sec execute 147 sec latency 1094.669 ms 2024-06-03T01:16:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:20 smithi107 ceph-mon[36622]: pgmap v182: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 397 KiB/s wr, 78 op/s 2024-06-03T01:16:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:20 smithi094 ceph-mon[32119]: pgmap v182: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 397 KiB/s wr, 78 op/s 2024-06-03T01:16:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:20 smithi179 ceph-mon[37041]: pgmap v182: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 397 KiB/s wr, 78 op/s 2024-06-03T01:16:21.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 135575 2.21 MB/sec execute 148 sec latency 2052.358 ms 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2335904594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 msgr2=0x7fe800109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2335904594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 0x7fe800109dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009930 tx=0x7fe7f002a420 comp rx=0 tx=0).stop 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2335904594 shutdown_connections 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2335904594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8001043a0 0x7fe8001108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2335904594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 0x7fe800103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2335904594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 0x7fe800109dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2335904594 >> 172.21.15.94:0/2335904594 conn(0x7fe8000ff4b0 msgr2=0x7fe8001018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2335904594 shutdown_connections 2024-06-03T01:16:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.136+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2335904594 wait complete. 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.137+0000 7fe7ffe00640 1 Processor -- start 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.137+0000 7fe7ffe00640 1 -- start start 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 0x7fe80019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7ffe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8001043a0 0x7fe80019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7ffe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 0x7fe80019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7ffe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113dd0 con 0x7fe8001043a0 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7ffe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113c50 con 0x7fe8001039e0 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7ffe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113f50 con 0x7fe8001099f0 2024-06-03T01:16:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 0x7fe80019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 0x7fe80019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:59036/0 (socket says 172.21.15.94:59036) 2024-06-03T01:16:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7ff400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 0x7fe80019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 -- 172.21.15.94:0/2310912266 learned_addr learned my addr 172.21.15.94:0/2310912266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7f6000640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8001043a0 0x7fe80019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 -- 172.21.15.94:0/2310912266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 msgr2=0x7fe80019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 0x7fe80019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 -- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8001043a0 msgr2=0x7fe80019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8001043a0 0x7fe80019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.138+0000 7fe7fea00640 1 -- 172.21.15.94:0/2310912266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80019f2e0 con 0x7fe8001039e0 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.139+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 0x7fe80019a2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009230 tx=0x7fe7f0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.139+0000 7fe7f7e00640 1 -- 172.21.15.94:0/2310912266 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f003f030 con 0x7fe8001039e0 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.139+0000 7fe7f7e00640 1 -- 172.21.15.94:0/2310912266 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f0003ad0 con 0x7fe8001039e0 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.139+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80019f570 con 0x7fe8001039e0 2024-06-03T01:16:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.139+0000 7fe7f7e00640 1 -- 172.21.15.94:0/2310912266 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0035070 con 0x7fe8001039e0 2024-06-03T01:16:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.139+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8001a6e60 con 0x7fe8001039e0 2024-06-03T01:16:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.143+0000 7fe80842d740 1 Processor -- start 2024-06-03T01:16:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.143+0000 7fe80842d740 1 -- start start 2024-06-03T01:16:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.143+0000 7fe7f7e00640 1 -- 172.21.15.94:0/2310912266 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe7f0042030 con 0x7fe8001039e0 2024-06-03T01:16:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe80842d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c018cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe80842d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c019680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe80842d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c026370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe80842d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b36bf7d420 con 0x55b36c023f80 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe80842d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b36bf24c50 con 0x55b36c01a770 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe80842d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b36bf793e0 con 0x55b36c019200 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c019680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe7ff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c018cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c019680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51030/0 (socket says 172.21.15.94:51030) 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe7fea00640 1 -- 172.21.15.94:0/2286647572 learned_addr learned my addr 172.21.15.94:0/2286647572 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe7f6000640 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c026370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.144+0000 7fe7fea00640 1 -- 172.21.15.94:0/2286647572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 msgr2=0x55b36c018cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c018cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7fea00640 1 -- 172.21.15.94:0/2286647572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 msgr2=0x55b36c026370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c026370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7fea00640 1 -- 172.21.15.94:0/2286647572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36bc91210 con 0x55b36c019200 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7f6000640 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c026370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c019680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ed070 tx=0x7fe7f00f3f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5090ba155ac979 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7ef400640 1 -- 172.21.15.94:0/2286647572 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00f6be0 con 0x55b36c019200 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7f7e00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe7e00a2cf0 0x7fe7e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7ef400640 1 -- 172.21.15.94:0/2286647572 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f00f6d80 con 0x55b36c019200 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.145+0000 7fe7ef400640 1 -- 172.21.15.94:0/2286647572 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0100630 con 0x55b36c019200 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe7f7e00640 1 -- 172.21.15.94:0/2310912266 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe7f00efd40 con 0x7fe8001039e0 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe7ff400640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe7e00a2cf0 0x7fe7e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe80842d740 1 -- 172.21.15.94:0/2286647572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 msgr2=0x55b36c019680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe80842d740 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c019680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ed070 tx=0x7fe7f00f3f80 comp rx=0 tx=0).stop 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe80842d740 1 -- 172.21.15.94:0/2286647572 shutdown_connections 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe80842d740 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c026370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe80842d740 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c019680 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe80842d740 1 --2- 172.21.15.94:0/2286647572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c018cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.146+0000 7fe80842d740 1 -- 172.21.15.94:0/2286647572 >> 172.21.15.94:0/2286647572 conn(0x55b36bfe1f80 msgr2=0x55b36c0157f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.147+0000 7fe7ff400640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe7e00a2cf0 0x7fe7e00a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8005fd0 tx=0x7fe7f8009500 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.147+0000 7fe80842d740 1 -- 172.21.15.94:0/2286647572 shutdown_connections 2024-06-03T01:16:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.147+0000 7fe80842d740 1 -- 172.21.15.94:0/2286647572 wait complete. 2024-06-03T01:16:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.147+0000 7fe80842d740 1 -- start start 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.147+0000 7fe80842d740 1 Processor -- start 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe80842d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c020f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe80842d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c0214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe80842d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c023f80 0x55b36c021a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe80842d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b36c026a70 con 0x55b36c019200 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe80842d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b36bf380b0 con 0x55b36c01a770 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe80842d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b36c026bf0 con 0x55b36c023f80 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe80842d740 10 client.0 _ll_register_callbacks cb 0x55b36bf23f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c020f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe7f6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c0214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe7ff400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c023f80 0x55b36c021a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c020f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55830/0 (socket says 172.21.15.94:55830) 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.148+0000 7fe7f6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c0214d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:59054/0 (socket says 172.21.15.94:59054) 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7fea00640 1 -- 172.21.15.94:0/211125958 learned_addr learned my addr 172.21.15.94:0/211125958 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7ff400640 1 -- 172.21.15.94:0/211125958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 msgr2=0x55b36c0214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7ff400640 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c0214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7ff400640 1 -- 172.21.15.94:0/211125958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 msgr2=0x55b36c020f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7ff400640 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c020f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7ff400640 1 -- 172.21.15.94:0/211125958 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36bd5d620 con 0x55b36c023f80 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7ff400640 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c023f80 0x55b36c021a10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8011dd0 tx=0x7fe7f8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb71d723f0037076 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.149+0000 7fe7e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe7e7e00640 1 -- 172.21.15.94:0/211125958 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8004b20 con 0x55b36c023f80 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f00f6d80 con 0x55b36c023f80 2024-06-03T01:16:21.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe7e7e00640 1 -- 172.21.15.94:0/211125958 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe7f8004cc0 con 0x55b36c023f80 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe7e7e00640 1 -- 172.21.15.94:0/211125958 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe7f800f810 con 0x55b36c023f80 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe80842d740 10 client.25792 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b36c032ec0 con 0x55b36c023f80 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe7e7e00640 1 -- 172.21.15.94:0/211125958 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f00f6d80 con 0x55b36c023f80 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.150+0000 7fe7e7e00640 1 -- 172.21.15.94:0/211125958 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe7f8026070 con 0x55b36c023f80 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.151+0000 7fe80842d740 10 client.25792 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.151+0000 7fe80842d740 10 client.25792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.151+0000 7fe80842d740 20 client.25792 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.151+0000 7fe80842d740 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c0343f0 0x55b36c0367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.151+0000 7fe80842d740 4 client.25792 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.151+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b36bf8ab50 con 0x55b36c0343f0 2024-06-03T01:16:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.151+0000 7fe7fea00640 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c0343f0 0x55b36c0367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.152+0000 7fe7fea00640 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c0343f0 0x55b36c0367e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfadba6eeea2169a server_cookie=a267ca3e8e812558 in_seq=0 out_seq=0 2024-06-03T01:16:21.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.152+0000 7fe7e7e00640 10 client.25792 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe7e7e00640 1 -- 172.21.15.94:0/211125958 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b36bf8ab50 con 0x55b36c0343f0 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe7e7e00640 10 client.25792 handle_command_reply: tid=0 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 client.25792 shutdown 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c023f80 msgr2=0x55b36c021a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c023f80 0x55b36c021a10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8011dd0 tx=0x7fe7f8004ab0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 shutdown_connections 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c023f80 0x55b36c021a10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c0343f0 0x55b36c0367e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c0214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 --2- 172.21.15.94:0/211125958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c020f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 >> 172.21.15.94:0/211125958 conn(0x55b36bfe1f80 msgr2=0x55b36c01b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe7e7e00640 0 client.25792 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 shutdown_connections 2024-06-03T01:16:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 1 -- 172.21.15.94:0/211125958 wait complete. 2024-06-03T01:16:21.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.153+0000 7fe80842d740 20 client.25792 trim_cache size 0 max 16384 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 Processor -- start 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 -- start start 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c021660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c03c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c03c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b0e70 con 0x55b36c023f80 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b29b0 con 0x55b36c01a770 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.171+0000 7fe80842d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b1540 con 0x55b36c019200 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c021660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7f6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c03c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c021660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51042/0 (socket says 172.21.15.94:51042) 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 -- 172.21.15.94:0/2687595309 learned_addr learned my addr 172.21.15.94:0/2687595309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7ff400640 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c03c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 -- 172.21.15.94:0/2687595309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 msgr2=0x55b36c03c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c03c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 -- 172.21.15.94:0/2687595309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 msgr2=0x55b36c03c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c03c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 -- 172.21.15.94:0/2687595309 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36c010fd0 con 0x55b36c019200 2024-06-03T01:16:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c021660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00f68b0 tx=0x7fe7f00f4d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb19283c94f5c3f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.172+0000 7fe7eea00640 1 -- 172.21.15.94:0/2687595309 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0049070 con 0x55b36c019200 2024-06-03T01:16:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe7eea00640 1 -- 172.21.15.94:0/2687595309 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f003b460 con 0x55b36c019200 2024-06-03T01:16:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe7eea00640 1 -- 172.21.15.94:0/2687595309 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00c02b0 con 0x55b36c019200 2024-06-03T01:16:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe80842d740 1 -- 172.21.15.94:0/2687595309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 msgr2=0x55b36c021660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe80842d740 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c021660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00f68b0 tx=0x7fe7f00f4d60 comp rx=0 tx=0).stop 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe80842d740 1 -- 172.21.15.94:0/2687595309 shutdown_connections 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe80842d740 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c023f80 0x55b36c03c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe80842d740 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c01a770 0x55b36c03c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe80842d740 1 --2- 172.21.15.94:0/2687595309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c019200 0x55b36c021660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.173+0000 7fe80842d740 1 -- 172.21.15.94:0/2687595309 >> 172.21.15.94:0/2687595309 conn(0x55b36bfe1f80 msgr2=0x55b36c020f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.174+0000 7fe80842d740 1 -- 172.21.15.94:0/2687595309 shutdown_connections 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.174+0000 7fe80842d740 1 -- 172.21.15.94:0/2687595309 wait complete. 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.174+0000 7fe80842d740 1 -- start start 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.174+0000 7fe80842d740 1 Processor -- start 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe80842d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c02c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe80842d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c01a770 0x55b36c0279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe80842d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c023f80 0x55b36c027f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe80842d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b22e0 con 0x55b36c019200 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe80842d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001b1c10 con 0x55b36c023f80 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe80842d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001aee50 con 0x55b36c01a770 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe80842d740 10 client.0 _ll_register_callbacks cb 0x7fe7f8009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c02c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7f6000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c01a770 0x55b36c0279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7ff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c023f80 0x55b36c027f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7f6000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c01a770 0x55b36c0279d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51048/0 (socket says 172.21.15.94:51048) 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7ff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c023f80 0x55b36c027f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:59072/0 (socket says 172.21.15.94:59072) 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7f6000640 1 -- 172.21.15.94:0/2833692770 learned_addr learned my addr 172.21.15.94:0/2833692770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7fea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c02c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55854/0 (socket says 172.21.15.94:55854) 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7ff400640 1 -- 172.21.15.94:0/2833692770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c01a770 msgr2=0x55b36c0279d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7ff400640 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c01a770 0x55b36c0279d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7ff400640 1 -- 172.21.15.94:0/2833692770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 msgr2=0x55b36c02c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7ff400640 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c02c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.175+0000 7fe7ff400640 1 -- 172.21.15.94:0/2833692770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36c028740 con 0x55b36c023f80 2024-06-03T01:16:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe7ff400640 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c023f80 0x55b36c027f10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8000f80 tx=0x7fe7f8004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d7ea1e133724420 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe7e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:16:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe7e7e00640 1 -- 172.21.15.94:0/2833692770 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8003d10 con 0x55b36c023f80 2024-06-03T01:16:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe7e7e00640 1 -- 172.21.15.94:0/2833692770 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fe7f801e640 con 0x55b36c023f80 2024-06-03T01:16:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b36bf84ea0 con 0x55b36c023f80 2024-06-03T01:16:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe7e7e00640 1 -- 172.21.15.94:0/2833692770 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe7f8042050 con 0x55b36c023f80 2024-06-03T01:16:21.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe80842d740 10 client.25781 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.176+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b36c09f0f0 con 0x55b36c023f80 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe7e7e00640 1 -- 172.21.15.94:0/2833692770 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b36bf84ea0 con 0x55b36c023f80 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe7e7e00640 1 -- 172.21.15.94:0/2833692770 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe7f8026050 con 0x55b36c023f80 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe80842d740 10 client.25781 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe80842d740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe80842d740 20 client.25781 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe80842d740 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c09f8f0 0x55b36c0a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe80842d740 4 client.25781 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b36bf027a0 con 0x55b36c09f8f0 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.177+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c09f8f0 0x55b36c0a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.178+0000 7fe7fea00640 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c09f8f0 0x55b36c0a1d10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52387abaab554c7 server_cookie=a18af744ceb63a31 in_seq=0 out_seq=0 2024-06-03T01:16:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.178+0000 7fe7e7e00640 10 client.25781 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe7e7e00640 1 -- 172.21.15.94:0/2833692770 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b36bf027a0 con 0x55b36c09f8f0 2024-06-03T01:16:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe7e7e00640 10 client.25781 handle_command_reply: tid=0 2024-06-03T01:16:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 client.25781 shutdown 2024-06-03T01:16:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c023f80 msgr2=0x55b36c027f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c023f80 0x55b36c027f10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8000f80 tx=0x7fe7f8004a10 comp rx=0 tx=0).stop 2024-06-03T01:16:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 shutdown_connections 2024-06-03T01:16:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b36c09f8f0 0x55b36c0a1d10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b36c023f80 0x55b36c027f10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe7e7e00640 0 client.25781 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b36c01a770 0x55b36c0279d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 --2- 172.21.15.94:0/2833692770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b36c019200 0x55b36c02c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.216+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 >> 172.21.15.94:0/2833692770 conn(0x55b36bfe1f80 msgr2=0x55b36c0217e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.217+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 shutdown_connections 2024-06-03T01:16:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.217+0000 7fe80842d740 1 -- 172.21.15.94:0/2833692770 wait complete. 2024-06-03T01:16:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.217+0000 7fe80842d740 20 client.25781 trim_cache size 0 max 16384 2024-06-03T01:16:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:16:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:16:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe7e00a2cf0 msgr2=0x7fe7e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe7e00a2cf0 0x7fe7e00a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8005fd0 tx=0x7fe7f8009500 comp rx=0 tx=0).stop 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 msgr2=0x7fe80019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 0x7fe80019a2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009230 tx=0x7fe7f0004270 comp rx=0 tx=0).stop 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 shutdown_connections 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fe7e00a2cf0 0x7fe7e00a51b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8001099f0 0x7fe80019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8001043a0 0x7fe80019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 --2- 172.21.15.94:0/2310912266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8001039e0 0x7fe80019a2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 >> 172.21.15.94:0/2310912266 conn(0x7fe8000ff4b0 msgr2=0x7fe8000ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 shutdown_connections 2024-06-03T01:16:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:21.218+0000 7fe7ffe00640 1 -- 172.21.15.94:0/2310912266 wait complete. 2024-06-03T01:16:22.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 137258 2.24 MB/sec execute 149 sec latency 50.317 ms 2024-06-03T01:16:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:22 smithi107 ceph-mon[36622]: pgmap v183: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 397 KiB/s wr, 78 op/s 2024-06-03T01:16:22.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:22 smithi094 ceph-mon[32119]: pgmap v183: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 397 KiB/s wr, 78 op/s 2024-06-03T01:16:22.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:22 smithi179 ceph-mon[37041]: pgmap v183: 129 pgs: 129 active+clean; 61 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 397 KiB/s wr, 78 op/s 2024-06-03T01:16:23.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 138597 2.30 MB/sec execute 150 sec latency 22.749 ms 2024-06-03T01:16:24.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139090 2.29 MB/sec execute 151 sec latency 183.466 ms 2024-06-03T01:16:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:24 smithi094 ceph-mon[32119]: pgmap v184: 129 pgs: 129 active+clean; 59 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-03T01:16:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:24 smithi179 ceph-mon[37041]: pgmap v184: 129 pgs: 129 active+clean; 59 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-03T01:16:24.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:24 smithi107 ceph-mon[36622]: pgmap v184: 129 pgs: 129 active+clean; 59 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-03T01:16:25.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139090 2.28 MB/sec execute 152 sec latency 1183.563 ms 2024-06-03T01:16:26.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139109 2.26 MB/sec execute 153 sec latency 2147.481 ms 2024-06-03T01:16:26.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:26 smithi094 ceph-mon[32119]: pgmap v185: 129 pgs: 129 active+clean; 59 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-03T01:16:26.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:26 smithi179 ceph-mon[37041]: pgmap v185: 129 pgs: 129 active+clean; 59 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-03T01:16:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:26 smithi107 ceph-mon[36622]: pgmap v185: 129 pgs: 129 active+clean; 59 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-03T01:16:27.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139158 2.25 MB/sec execute 154 sec latency 950.999 ms 2024-06-03T01:16:28.089 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139158 2.23 MB/sec execute 155 sec latency 1951.123 ms 2024-06-03T01:16:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:28 smithi094 ceph-mon[32119]: pgmap v186: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-06-03T01:16:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:28 smithi179 ceph-mon[37041]: pgmap v186: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-06-03T01:16:28.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:28 smithi107 ceph-mon[36622]: pgmap v186: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-06-03T01:16:29.090 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139158 2.22 MB/sec execute 156 sec latency 2951.286 ms 2024-06-03T01:16:30.090 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139158 2.21 MB/sec execute 157 sec latency 3951.386 ms 2024-06-03T01:16:30.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:30 smithi094 ceph-mon[32119]: pgmap v187: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-06-03T01:16:30.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:30 smithi179 ceph-mon[37041]: pgmap v187: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-06-03T01:16:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:30 smithi107 ceph-mon[36622]: pgmap v187: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-06-03T01:16:31.090 INFO:tasks.workunit.client.0.smithi094.stdout: 1 139180 2.19 MB/sec execute 158 sec latency 4912.659 ms 2024-06-03T01:16:32.090 INFO:tasks.workunit.client.0.smithi094.stdout: 1 140803 2.22 MB/sec execute 159 sec latency 14.815 ms 2024-06-03T01:16:32.333 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:32 smithi107 ceph-mon[36622]: pgmap v188: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-06-03T01:16:32.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:32 smithi094 ceph-mon[32119]: pgmap v188: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-06-03T01:16:32.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:32 smithi179 ceph-mon[37041]: pgmap v188: 129 pgs: 129 active+clean; 60 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-06-03T01:16:33.090 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142006 2.27 MB/sec execute 160 sec latency 47.095 ms 2024-06-03T01:16:34.090 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142516 2.27 MB/sec execute 161 sec latency 16.335 ms 2024-06-03T01:16:34.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:34 smithi094 ceph-mon[32119]: pgmap v189: 129 pgs: 129 active+clean; 61 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-06-03T01:16:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:34 smithi179 ceph-mon[37041]: pgmap v189: 129 pgs: 129 active+clean; 61 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-06-03T01:16:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2640381972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c112840 msgr2=0x7fd69c114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2640381972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c112840 0x7fd69c114c30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd694009be0 tx=0x7fd69402a6c0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2640381972 shutdown_connections 2024-06-03T01:16:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2640381972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c112840 0x7fd69c114c30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2640381972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd69c0767f0 0x7fd69c112300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2640381972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69c075ed0 0x7fd69c0762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2640381972 >> 172.21.15.94:0/2640381972 conn(0x7fd69c0ff6d0 msgr2=0x7fd69c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2640381972 shutdown_connections 2024-06-03T01:16:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.359+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2640381972 wait complete. 2024-06-03T01:16:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.360+0000 7fd6a0c00640 1 Processor -- start 2024-06-03T01:16:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.360+0000 7fd6a0c00640 1 -- start start 2024-06-03T01:16:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd6a0c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c075ed0 0x7fd69c1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd6a0c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd69c0767f0 0x7fd69c1a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd6a0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69c112840 0x7fd69c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd6a0c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c118310 con 0x7fd69c075ed0 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd6a0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c118190 con 0x7fd69c112840 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd6a0c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c118490 con 0x7fd69c0767f0 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c075ed0 0x7fd69c1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c075ed0 0x7fd69c1a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47398/0 (socket says 172.21.15.94:47398) 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699e00640 1 -- 172.21.15.94:0/2442516265 learned_addr learned my addr 172.21.15.94:0/2442516265 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd69a800640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69c112840 0x7fd69c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699400640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd69c0767f0 0x7fd69c1a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699e00640 1 -- 172.21.15.94:0/2442516265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd69c0767f0 msgr2=0x7fd69c1a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699e00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd69c0767f0 0x7fd69c1a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699e00640 1 -- 172.21.15.94:0/2442516265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69c112840 msgr2=0x7fd69c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.361+0000 7fd699e00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69c112840 0x7fd69c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.362+0000 7fd699e00640 1 -- 172.21.15.94:0/2442516265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c19eef0 con 0x7fd69c075ed0 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.362+0000 7fd699e00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c075ed0 0x7fd69c1a4570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e9f0 tx=0x7fd68c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.362+0000 7fd692a00640 1 -- 172.21.15.94:0/2442516265 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00cd80 con 0x7fd69c075ed0 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.362+0000 7fd692a00640 1 -- 172.21.15.94:0/2442516265 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68c010910 con 0x7fd69c075ed0 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.362+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69c19f1e0 con 0x7fd69c075ed0 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.362+0000 7fd692a00640 1 -- 172.21.15.94:0/2442516265 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c018990 con 0x7fd69c075ed0 2024-06-03T01:16:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.362+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69c1ab4f0 con 0x7fd69c075ed0 2024-06-03T01:16:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.365+0000 7fd6a3b96740 1 Processor -- start 2024-06-03T01:16:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd6a3b96740 1 -- start start 2024-06-03T01:16:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd692a00640 1 -- 172.21.15.94:0/2442516265 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd68c010ac0 con 0x7fd69c075ed0 2024-06-03T01:16:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb0fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb1adc0 0x55562eb1d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55562ea74420 con 0x55562eb10040 2024-06-03T01:16:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55562ea1bc50 con 0x55562eb1adc0 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55562ea703e0 con 0x55562eb115b0 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd69a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb0fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd699e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd69a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb0fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:52576/0 (socket says 172.21.15.94:52576) 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd69a800640 1 -- 172.21.15.94:0/3401829251 learned_addr learned my addr 172.21.15.94:0/3401829251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.366+0000 7fd699400640 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb1adc0 0x55562eb1d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd699400640 1 -- 172.21.15.94:0/3401829251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 msgr2=0x55562eb0fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd699400640 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb0fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd699400640 1 -- 172.21.15.94:0/3401829251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 msgr2=0x55562eb104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd699400640 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb104c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd699400640 1 -- 172.21.15.94:0/3401829251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55562e788210 con 0x55562eb1adc0 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd699400640 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb1adc0 0x55562eb1d1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd688005fd0 tx=0x7fd688011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72a00d9109392ba server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd687000640 1 -- 172.21.15.94:0/3401829251 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688009db0 con 0x55562eb1adc0 2024-06-03T01:16:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd687000640 1 -- 172.21.15.94:0/3401829251 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6880035f0 con 0x55562eb1adc0 2024-06-03T01:16:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd687000640 1 -- 172.21.15.94:0/3401829251 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688019670 con 0x55562eb1adc0 2024-06-03T01:16:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.367+0000 7fd692a00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd6800a2da0 0x7fd6800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd692a00640 1 -- 172.21.15.94:0/2442516265 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fd68c0cfb00 con 0x7fd69c075ed0 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd69a800640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd6800a2da0 0x7fd6800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3401829251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb1adc0 msgr2=0x55562eb1d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb1adc0 0x55562eb1d1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd688005fd0 tx=0x7fd688011810 comp rx=0 tx=0).stop 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3401829251 shutdown_connections 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb1adc0 0x55562eb1d1b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3401829251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb0fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3401829251 >> 172.21.15.94:0/3401829251 conn(0x55562ead8f80 msgr2=0x55562eb0c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.368+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3401829251 shutdown_connections 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.369+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3401829251 wait complete. 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.369+0000 7fd69a800640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd6800a2da0 0x7fd6800a5260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd69c06ad80 tx=0x7fd6940098a0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.369+0000 7fd6a3b96740 1 -- start start 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.369+0000 7fd6a3b96740 1 Processor -- start 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb115b0 0x55562eb18320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb18860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55562eb1db90 con 0x55562eb115b0 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55562ea2f0b0 con 0x55562eb10040 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55562eb1dd10 con 0x55562eb1adc0 2024-06-03T01:16:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd6a3b96740 10 client.0 _ll_register_callbacks cb 0x55562ea1af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd699400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb115b0 0x55562eb18320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd69a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb18860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd699e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb17de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd69a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb18860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:52584/0 (socket says 172.21.15.94:52584) 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.370+0000 7fd69a800640 1 -- 172.21.15.94:0/3598901383 learned_addr learned my addr 172.21.15.94:0/3598901383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd69a800640 1 -- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 msgr2=0x55562eb17de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd69a800640 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb17de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd69a800640 1 -- 172.21.15.94:0/3598901383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb115b0 msgr2=0x55562eb18320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd69a800640 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb115b0 0x55562eb18320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd69a800640 1 -- 172.21.15.94:0/3598901383 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562e854620 con 0x55562eb1adc0 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd699e00640 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb17de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd69a800640 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb18860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd694006010 tx=0x7fd69402a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a495a2d825985073 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd67b400640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd67b400640 1 -- 172.21.15.94:0/3598901383 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694050020 con 0x55562eb1adc0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6880035f0 con 0x55562eb1adc0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd67b400640 1 -- 172.21.15.94:0/3598901383 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fd694003b70 con 0x55562eb1adc0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.371+0000 7fd67b400640 1 -- 172.21.15.94:0/3598901383 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd69405b050 con 0x55562eb1adc0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 10 client.25810 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55562eb29fb0 con 0x55562eb1adc0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd67b400640 1 -- 172.21.15.94:0/3598901383 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6880035f0 con 0x55562eb1adc0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd67b400640 1 -- 172.21.15.94:0/3598901383 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd69403c040 con 0x55562eb1adc0 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 10 client.25810 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 10 client.25810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 20 client.25810 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb2b470 0x55562eb2d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 4 client.25810 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:16:34.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.372+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55562ea81b50 con 0x55562eb2b470 2024-06-03T01:16:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.373+0000 7fd699e00640 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb2b470 0x55562eb2d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.373+0000 7fd699e00640 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb2b470 0x55562eb2d8c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f29bf15dedae692 server_cookie=bbd3dab8342c70b0 in_seq=0 out_seq=0 2024-06-03T01:16:34.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.373+0000 7fd67b400640 10 client.25810 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd67b400640 1 -- 172.21.15.94:0/3598901383 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55562ea81b50 con 0x55562eb2b470 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd67b400640 10 client.25810 handle_command_reply: tid=0 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 client.25810 shutdown 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 msgr2=0x55562eb18860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb18860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd694006010 tx=0x7fd69402a6c0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 shutdown_connections 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb18860 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb115b0 0x55562eb18320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb2b470 0x55562eb2d8c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/3598901383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb17de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 >> 172.21.15.94:0/3598901383 conn(0x55562ead8f80 msgr2=0x55562eb124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd67b400640 0 client.25810 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.374+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 shutdown_connections 2024-06-03T01:16:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.375+0000 7fd6a3b96740 1 -- 172.21.15.94:0/3598901383 wait complete. 2024-06-03T01:16:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.375+0000 7fd6a3b96740 20 client.25810 trim_cache size 0 max 16384 2024-06-03T01:16:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 Processor -- start 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 -- start start 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb115b0 0x55562eb33710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb11c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1b5400 con 0x55562eb10040 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1b6f40 con 0x55562eb115b0 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.391+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1b5ad0 con 0x55562eb1adc0 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd699e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd69a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb11c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd699e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb331d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47420/0 (socket says 172.21.15.94:47420) 2024-06-03T01:16:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd699e00640 1 -- 172.21.15.94:0/1633437108 learned_addr learned my addr 172.21.15.94:0/1633437108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd69a800640 1 -- 172.21.15.94:0/1633437108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb115b0 msgr2=0x55562eb33710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd69a800640 1 --2- 172.21.15.94:0/1633437108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb115b0 0x55562eb33710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd69a800640 1 -- 172.21.15.94:0/1633437108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 msgr2=0x55562eb331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd69a800640 1 --2- 172.21.15.94:0/1633437108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb331d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd69a800640 1 -- 172.21.15.94:0/1633437108 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55562eb10b10 con 0x55562eb1adc0 2024-06-03T01:16:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.392+0000 7fd69a800640 1 --2- 172.21.15.94:0/1633437108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb11c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd694009be0 tx=0x7fd694005cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7e7898804521953 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd686600640 1 -- 172.21.15.94:0/1633437108 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694064070 con 0x55562eb1adc0 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd686600640 1 -- 172.21.15.94:0/1633437108 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd694037040 con 0x55562eb1adc0 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd686600640 1 -- 172.21.15.94:0/1633437108 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694050070 con 0x55562eb1adc0 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 -- 172.21.15.94:0/1633437108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 msgr2=0x55562eb11c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/1633437108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb11c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd694009be0 tx=0x7fd694005cb0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 -- 172.21.15.94:0/1633437108 shutdown_connections 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/1633437108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb1adc0 0x55562eb11c10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/1633437108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb115b0 0x55562eb33710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/1633437108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb10040 0x55562eb331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 -- 172.21.15.94:0/1633437108 >> 172.21.15.94:0/1633437108 conn(0x55562ead8f80 msgr2=0x55562eb166b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 -- 172.21.15.94:0/1633437108 shutdown_connections 2024-06-03T01:16:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.393+0000 7fd6a3b96740 1 -- 172.21.15.94:0/1633437108 wait complete. 2024-06-03T01:16:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.394+0000 7fd6a3b96740 1 -- start start 2024-06-03T01:16:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.394+0000 7fd6a3b96740 1 Processor -- start 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.394+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb1e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.394+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb1e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.394+0000 7fd6a3b96740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb1adc0 0x55562eb94190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1b6870 con 0x55562eb1adc0 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1b61a0 con 0x55562eb10040 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd6a3b96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1b33e0 con 0x55562eb115b0 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd6a3b96740 10 client.0 _ll_register_callbacks cb 0x7fd694043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb1adc0 0x55562eb94190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd699400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb1e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd699400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb1e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:52602/0 (socket says 172.21.15.94:52602) 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb1adc0 0x55562eb94190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47436/0 (socket says 172.21.15.94:47436) 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd699400640 1 -- 172.21.15.94:0/2070184426 learned_addr learned my addr 172.21.15.94:0/2070184426 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd699e00640 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb1e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 -- 172.21.15.94:0/2070184426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 msgr2=0x55562eb1e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb1e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 -- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 msgr2=0x55562eb1e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb1e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 -- 172.21.15.94:0/2070184426 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55562eb946d0 con 0x55562eb1adc0 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd69a800640 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb1adc0 0x55562eb94190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd694031670 tx=0x7fd694031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad927aa2a11b59b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.395+0000 7fd67b400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd67b400640 1 -- 172.21.15.94:0/2070184426 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694064070 con 0x55562eb1adc0 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd67b400640 1 -- 172.21.15.94:0/2070184426 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fd694003970 con 0x55562eb1adc0 2024-06-03T01:16:34.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd694037040 con 0x55562eb1adc0 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd67b400640 1 -- 172.21.15.94:0/2070184426 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd69406f050 con 0x55562eb1adc0 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd6a3b96740 10 client.15678 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55562eb958e0 con 0x55562eb1adc0 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd67b400640 1 -- 172.21.15.94:0/2070184426 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd694037040 con 0x55562eb1adc0 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd67b400640 1 -- 172.21.15.94:0/2070184426 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd69404c480 con 0x55562eb1adc0 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd6a3b96740 10 client.15678 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd6a3b96740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.396+0000 7fd6a3b96740 20 client.15678 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.397+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb95e90 0x55562eb98280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.397+0000 7fd6a3b96740 4 client.15678 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.397+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55562ea7bea0 con 0x55562eb95e90 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.397+0000 7fd699e00640 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb95e90 0x55562eb98280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.397+0000 7fd699e00640 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb95e90 0x55562eb98280 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866f440c4bfe24e8 server_cookie=2836fa5bc201dd7f in_seq=0 out_seq=0 2024-06-03T01:16:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.397+0000 7fd67b400640 10 client.15678 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd67b400640 1 -- 172.21.15.94:0/2070184426 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55562ea7bea0 con 0x55562eb95e90 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd67b400640 10 client.15678 handle_command_reply: tid=0 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd6a3b96740 1 client.15678 shutdown 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb1adc0 msgr2=0x55562eb94190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb1adc0 0x55562eb94190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd694031670 tx=0x7fd694031b10 comp rx=0 tx=0).stop 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 shutdown_connections 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55562eb1adc0 0x55562eb94190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55562eb115b0 0x55562eb1e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.438+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55562eb95e90 0x55562eb98280 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.439+0000 7fd6a3b96740 1 --2- 172.21.15.94:0/2070184426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55562eb10040 0x55562eb1e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.439+0000 7fd67b400640 0 client.15678 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.439+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 >> 172.21.15.94:0/2070184426 conn(0x55562ead8f80 msgr2=0x55562eb11c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.439+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 shutdown_connections 2024-06-03T01:16:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.439+0000 7fd6a3b96740 1 -- 172.21.15.94:0/2070184426 wait complete. 2024-06-03T01:16:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.439+0000 7fd6a3b96740 20 client.15678 trim_cache size 0 max 16384 2024-06-03T01:16:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:16:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:16:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd6800a2da0 msgr2=0x7fd6800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd6800a2da0 0x7fd6800a5260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd69c06ad80 tx=0x7fd6940098a0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c075ed0 msgr2=0x7fd69c1a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c075ed0 0x7fd69c1a4570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e9f0 tx=0x7fd68c00eeb0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 shutdown_connections 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd6800a2da0 0x7fd6800a5260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd69c112840 0x7fd69c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd69c0767f0 0x7fd69c1a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 --2- 172.21.15.94:0/2442516265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd69c075ed0 0x7fd69c1a4570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 >> 172.21.15.94:0/2442516265 conn(0x7fd69c0ff6d0 msgr2=0x7fd69c1134b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 shutdown_connections 2024-06-03T01:16:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:34.440+0000 7fd6a0c00640 1 -- 172.21.15.94:0/2442516265 wait complete. 2024-06-03T01:16:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:34 smithi107 ceph-mon[36622]: pgmap v189: 129 pgs: 129 active+clean; 61 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-06-03T01:16:35.090 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142577 2.25 MB/sec execute 162 sec latency 971.131 ms 2024-06-03T01:16:36.091 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142594 2.24 MB/sec execute 163 sec latency 1935.722 ms 2024-06-03T01:16:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:36 smithi094 ceph-mon[32119]: pgmap v190: 129 pgs: 129 active+clean; 61 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 796 KiB/s wr, 75 op/s 2024-06-03T01:16:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:36 smithi179 ceph-mon[37041]: pgmap v190: 129 pgs: 129 active+clean; 61 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 796 KiB/s wr, 75 op/s 2024-06-03T01:16:36.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:36 smithi107 ceph-mon[36622]: pgmap v190: 129 pgs: 129 active+clean; 61 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 796 KiB/s wr, 75 op/s 2024-06-03T01:16:37.091 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142701 2.23 MB/sec execute 164 sec latency 857.733 ms 2024-06-03T01:16:38.091 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142701 2.21 MB/sec execute 165 sec latency 1857.812 ms 2024-06-03T01:16:38.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:38 smithi094 ceph-mon[32119]: pgmap v191: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-03T01:16:38.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:38 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:38 smithi179 ceph-mon[37041]: pgmap v191: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-03T01:16:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:38 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:38 smithi107 ceph-mon[36622]: pgmap v191: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-03T01:16:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:38 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:16:39.091 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142701 2.20 MB/sec execute 166 sec latency 2857.890 ms 2024-06-03T01:16:39.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:39 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:39.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:39 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:39.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:39 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:16:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:39 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:39 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:39 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:16:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:39 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:16:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:39 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:16:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:39 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:16:40.091 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142701 2.19 MB/sec execute 167 sec latency 3858.078 ms 2024-06-03T01:16:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:40 smithi094 ceph-mon[32119]: pgmap v192: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 639 KiB/s wr, 79 op/s 2024-06-03T01:16:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:40 smithi179 ceph-mon[37041]: pgmap v192: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 639 KiB/s wr, 79 op/s 2024-06-03T01:16:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:40 smithi107 ceph-mon[36622]: pgmap v192: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 639 KiB/s wr, 79 op/s 2024-06-03T01:16:41.091 INFO:tasks.workunit.client.0.smithi094.stdout: 1 142720 2.18 MB/sec execute 168 sec latency 4818.936 ms 2024-06-03T01:16:42.091 INFO:tasks.workunit.client.0.smithi094.stdout: 1 144313 2.20 MB/sec execute 169 sec latency 14.785 ms 2024-06-03T01:16:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:42 smithi094 ceph-mon[32119]: pgmap v193: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 639 KiB/s wr, 79 op/s 2024-06-03T01:16:42.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:42 smithi179 ceph-mon[37041]: pgmap v193: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 639 KiB/s wr, 79 op/s 2024-06-03T01:16:42.532 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:42 smithi107 ceph-mon[36622]: pgmap v193: 129 pgs: 129 active+clean; 59 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 639 KiB/s wr, 79 op/s 2024-06-03T01:16:43.092 INFO:tasks.workunit.client.0.smithi094.stdout: 1 145533 2.25 MB/sec execute 170 sec latency 32.148 ms 2024-06-03T01:16:44.092 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146061 2.25 MB/sec execute 171 sec latency 28.823 ms 2024-06-03T01:16:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:44 smithi094 ceph-mon[32119]: pgmap v194: 129 pgs: 129 active+clean; 60 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 960 KiB/s wr, 103 op/s 2024-06-03T01:16:44.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:44 smithi179 ceph-mon[37041]: pgmap v194: 129 pgs: 129 active+clean; 60 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 960 KiB/s wr, 103 op/s 2024-06-03T01:16:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:44 smithi107 ceph-mon[36622]: pgmap v194: 129 pgs: 129 active+clean; 60 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 960 KiB/s wr, 103 op/s 2024-06-03T01:16:45.092 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146126 2.23 MB/sec execute 172 sec latency 962.748 ms 2024-06-03T01:16:46.092 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146143 2.22 MB/sec execute 173 sec latency 1926.554 ms 2024-06-03T01:16:46.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:46 smithi094 ceph-mon[32119]: pgmap v195: 129 pgs: 129 active+clean; 60 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 633 KiB/s wr, 78 op/s 2024-06-03T01:16:46.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:46 smithi179 ceph-mon[37041]: pgmap v195: 129 pgs: 129 active+clean; 60 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 633 KiB/s wr, 78 op/s 2024-06-03T01:16:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:46 smithi107 ceph-mon[36622]: pgmap v195: 129 pgs: 129 active+clean; 60 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 633 KiB/s wr, 78 op/s 2024-06-03T01:16:47.092 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146182 2.21 MB/sec execute 174 sec latency 942.253 ms 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.607+0000 7fb357c00640 1 -- 172.21.15.94:0/1024782299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb350103e10 msgr2=0x7fb3501041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.607+0000 7fb357c00640 1 --2- 172.21.15.94:0/1024782299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb350103e10 0x7fb3501041f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb340009900 tx=0x7fb34002a410 comp rx=0 tx=0).stop 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.608+0000 7fb357c00640 1 -- 172.21.15.94:0/1024782299 shutdown_connections 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.608+0000 7fb357c00640 1 --2- 172.21.15.94:0/1024782299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350108ee0 0x7fb350110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.608+0000 7fb357c00640 1 --2- 172.21.15.94:0/1024782299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3501047c0 0x7fb350108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.608+0000 7fb357c00640 1 --2- 172.21.15.94:0/1024782299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb350103e10 0x7fb3501041f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.608+0000 7fb357c00640 1 -- 172.21.15.94:0/1024782299 >> 172.21.15.94:0/1024782299 conn(0x7fb3500ff6b0 msgr2=0x7fb350101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.608+0000 7fb357c00640 1 -- 172.21.15.94:0/1024782299 shutdown_connections 2024-06-03T01:16:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.608+0000 7fb357c00640 1 -- 172.21.15.94:0/1024782299 wait complete. 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 Processor -- start 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 -- start start 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb350103e10 0x7fb35019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350108ee0 0x7fb35019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35019ecf0 0x7fb35019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb350113f90 con 0x7fb350108ee0 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb350113e10 con 0x7fb350103e10 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb357c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb350114110 con 0x7fb35019ecf0 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb355200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb350103e10 0x7fb35019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb355200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb350103e10 0x7fb35019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33862/0 (socket says 172.21.15.94:33862) 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.609+0000 7fb355200640 1 -- 172.21.15.94:0/1368349455 learned_addr learned my addr 172.21.15.94:0/1368349455 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb347e00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350108ee0 0x7fb35019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35019ecf0 0x7fb35019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355200640 1 -- 172.21.15.94:0/1368349455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35019ecf0 msgr2=0x7fb35019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355200640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35019ecf0 0x7fb35019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355200640 1 -- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350108ee0 msgr2=0x7fb35019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355200640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350108ee0 0x7fb35019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355200640 1 -- 172.21.15.94:0/1368349455 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35019f7e0 con 0x7fb350103e10 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35019ecf0 0x7fb35019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb355200640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb350103e10 0x7fb35019a430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb340006010 tx=0x7fb340004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb34e000640 1 -- 172.21.15.94:0/1368349455 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34003f030 con 0x7fb350103e10 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.610+0000 7fb34e000640 1 -- 172.21.15.94:0/1368349455 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb340004440 con 0x7fb350103e10 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.611+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35019fa70 con 0x7fb350103e10 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.611+0000 7fb34e000640 1 -- 172.21.15.94:0/1368349455 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340035070 con 0x7fb350103e10 2024-06-03T01:16:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.611+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3501a71d0 con 0x7fb350103e10 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 Processor -- start 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb34e000640 1 -- 172.21.15.94:0/1368349455 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb340042030 con 0x7fb350103e10 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 -- start start 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa5cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5d040 0x55f19aa5d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f19a9c1420 con 0x55f19aa67dc0 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f19a968c50 con 0x55f19aa5e5b0 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.614+0000 7fb35ab98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f19a9bd3e0 con 0x55f19aa5d040 2024-06-03T01:16:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa5cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb347e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa5cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33868/0 (socket says 172.21.15.94:33868) 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 -- 172.21.15.94:0/3759196319 learned_addr learned my addr 172.21.15.94:0/3759196319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355200640 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5d040 0x55f19aa5d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 -- 172.21.15.94:0/3759196319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5d040 msgr2=0x55f19aa5d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5d040 0x55f19aa5d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 -- 172.21.15.94:0/3759196319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 msgr2=0x55f19aa6a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa6a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 -- 172.21.15.94:0/3759196319 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19a6d5210 con 0x55f19aa5e5b0 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb347e00640 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.615+0000 7fb355c00640 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa5cb00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3480117c0 tx=0x7fb348011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae30627d9c6570a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb34e000640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb3340a2da0 0x7fb3340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb346000640 1 -- 172.21.15.94:0/3759196319 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348003c30 con 0x55f19aa5e5b0 2024-06-03T01:16:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb346000640 1 -- 172.21.15.94:0/3759196319 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34800fe80 con 0x55f19aa5e5b0 2024-06-03T01:16:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb346000640 1 -- 172.21.15.94:0/3759196319 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480193e0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb34e000640 1 -- 172.21.15.94:0/1368349455 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fb3400efb60 con 0x7fb350103e10 2024-06-03T01:16:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb355200640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb3340a2da0 0x7fb3340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb35ab98740 1 -- 172.21.15.94:0/3759196319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 msgr2=0x55f19aa5cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.616+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa5cb00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3480117c0 tx=0x7fb348011c80 comp rx=0 tx=0).stop 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 -- 172.21.15.94:0/3759196319 shutdown_connections 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5d040 0x55f19aa5d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3759196319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa5cb00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb355200640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb3340a2da0 0x7fb3340a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55f19aa5fd10 tx=0x7fb3400f3610 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 -- 172.21.15.94:0/3759196319 >> 172.21.15.94:0/3759196319 conn(0x55f19aa25f80 msgr2=0x55f19aa59630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 -- 172.21.15.94:0/3759196319 shutdown_connections 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 -- 172.21.15.94:0/3759196319 wait complete. 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 -- start start 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.617+0000 7fb35ab98740 1 Processor -- start 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5d040 0x55f19aa64e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb35ab98740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f19aa6ab90 con 0x55f19aa67dc0 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb35ab98740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f19a97c0b0 con 0x55f19aa5d040 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb35ab98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f19aa6ad10 con 0x55f19aa5e5b0 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb35ab98740 10 client.0 _ll_register_callbacks cb 0x55f19a967f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb355c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5d040 0x55f19aa64e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb355c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5d040 0x55f19aa64e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:33872/0 (socket says 172.21.15.94:33872) 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb355c00640 1 -- 172.21.15.94:0/2746086079 learned_addr learned my addr 172.21.15.94:0/2746086079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb347e00640 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.618+0000 7fb355c00640 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb347e00640 1 -- 172.21.15.94:0/2746086079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5d040 msgr2=0x55f19aa64e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb347e00640 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5d040 0x55f19aa64e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb347e00640 1 -- 172.21.15.94:0/2746086079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 msgr2=0x55f19aa658f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb347e00640 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa658f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb347e00640 1 -- 172.21.15.94:0/2746086079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19a7a1620 con 0x55f19aa5e5b0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb347e00640 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa653b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55f19aa5dae0 tx=0x7fb338012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b32d62f8050c0a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb33ea00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb33ea00640 1 -- 172.21.15.94:0/2746086079 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338004030 con 0x55f19aa5e5b0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f19a9ceb50 con 0x55f19aa5e5b0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.619+0000 7fb33ea00640 1 -- 172.21.15.94:0/2746086079 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fb3380041d0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb33ea00640 1 -- 172.21.15.94:0/2746086079 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb338025050 con 0x55f19aa5e5b0 2024-06-03T01:16:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 10 client.25828 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f19aa77000 con 0x55f19aa5e5b0 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb33ea00640 1 -- 172.21.15.94:0/2746086079 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f19a9ceb50 con 0x55f19aa5e5b0 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb33ea00640 1 -- 172.21.15.94:0/2746086079 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb3380039d0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 10 client.25828 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 20 client.25828 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aa784c0 0x55f19aa7a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 4 client.25828 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.620+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f19a9c8ea0 con 0x55f19aa784c0 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.621+0000 7fb355c00640 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aa784c0 0x55f19aa7a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.621+0000 7fb355c00640 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aa784c0 0x55f19aa7a8b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa5a690181f8b43 server_cookie=cd52cf0fb9e82c14 in_seq=0 out_seq=0 2024-06-03T01:16:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.621+0000 7fb33ea00640 10 client.25828 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb33ea00640 1 -- 172.21.15.94:0/2746086079 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f19a9c8ea0 con 0x55f19aa784c0 2024-06-03T01:16:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb33ea00640 10 client.25828 handle_command_reply: tid=0 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 client.25828 shutdown 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 msgr2=0x55f19aa653b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa653b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55f19aa5dae0 tx=0x7fb338012b60 comp rx=0 tx=0).stop 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 shutdown_connections 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa67dc0 0x55f19aa658f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa653b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aa784c0 0x55f19aa7a8b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 --2- 172.21.15.94:0/2746086079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5d040 0x55f19aa64e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 >> 172.21.15.94:0/2746086079 conn(0x55f19aa25f80 msgr2=0x55f19aa5f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb33ea00640 0 client.25828 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 shutdown_connections 2024-06-03T01:16:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.622+0000 7fb35ab98740 1 -- 172.21.15.94:0/2746086079 wait complete. 2024-06-03T01:16:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.623+0000 7fb35ab98740 20 client.25828 trim_cache size 0 max 16384 2024-06-03T01:16:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.639+0000 7fb35ab98740 1 Processor -- start 2024-06-03T01:16:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.639+0000 7fb35ab98740 1 -- start start 2024-06-03T01:16:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa60260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa65710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa67dc0 0x55f19aa58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb35ab98740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501b0f00 con 0x55f19aa5d040 2024-06-03T01:16:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb35ab98740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501b2a40 con 0x55f19aa5e5b0 2024-06-03T01:16:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb35ab98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501b15d0 con 0x55f19aa67dc0 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb355c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa60260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb347e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa65710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb355c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa60260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38858/0 (socket says 172.21.15.94:38858) 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb355c00640 1 -- 172.21.15.94:0/1454219040 learned_addr learned my addr 172.21.15.94:0/1454219040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb347e00640 1 -- 172.21.15.94:0/1454219040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa67dc0 msgr2=0x55f19aa58f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb355c00640 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa67dc0 0x55f19aa58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb347e00640 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa67dc0 0x55f19aa58f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb347e00640 1 -- 172.21.15.94:0/1454219040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 msgr2=0x55f19aa60260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb347e00640 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa60260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb347e00640 1 -- 172.21.15.94:0/1454219040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19aa5fd40 con 0x55f19aa5e5b0 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.640+0000 7fb355c00640 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa67dc0 0x55f19aa58f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.641+0000 7fb347e00640 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa65710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb338012420 tx=0x7fb3380096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638e87aa36a0e381 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.641+0000 7fb345600640 1 -- 172.21.15.94:0/1454219040 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338022040 con 0x55f19aa5e5b0 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.641+0000 7fb345600640 1 -- 172.21.15.94:0/1454219040 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3380105b0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.641+0000 7fb345600640 1 -- 172.21.15.94:0/1454219040 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338005330 con 0x55f19aa5e5b0 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 -- 172.21.15.94:0/1454219040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 msgr2=0x55f19aa65710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa65710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb338012420 tx=0x7fb3380096a0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 -- 172.21.15.94:0/1454219040 shutdown_connections 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa67dc0 0x55f19aa58f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa5e5b0 0x55f19aa65710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 --2- 172.21.15.94:0/1454219040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa60260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 -- 172.21.15.94:0/1454219040 >> 172.21.15.94:0/1454219040 conn(0x55f19aa25f80 msgr2=0x55f19aa5f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 -- 172.21.15.94:0/1454219040 shutdown_connections 2024-06-03T01:16:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 -- 172.21.15.94:0/1454219040 wait complete. 2024-06-03T01:16:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 -- start start 2024-06-03T01:16:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.642+0000 7fb35ab98740 1 Processor -- start 2024-06-03T01:16:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb35ab98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa67dc0 0x55f19aa6efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb35ab98740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501b2370 con 0x55f19aa5d040 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb35ab98740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501b1ca0 con 0x55f19aa67dc0 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb35ab98740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501aeee0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb35ab98740 10 client.0 _ll_register_callbacks cb 0x7fb338009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb355c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb355c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38864/0 (socket says 172.21.15.94:38864) 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb355c00640 1 -- 172.21.15.94:0/3710250412 learned_addr learned my addr 172.21.15.94:0/3710250412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb347e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.643+0000 7fb355c00640 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa67dc0 0x55f19aa6efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb347e00640 1 -- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa67dc0 msgr2=0x55f19aa6efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb347e00640 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa67dc0 0x55f19aa6efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb347e00640 1 -- 172.21.15.94:0/3710250412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 msgr2=0x55f19aa687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb347e00640 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb347e00640 1 -- 172.21.15.94:0/3710250412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19aa6f4e0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb355c00640 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb347e00640 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa68ce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55f19aa5a140 tx=0x7fb338003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8ba6643e839ba0b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb33ea00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb33ea00640 1 -- 172.21.15.94:0/3710250412 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33800f8f0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb33ea00640 1 -- 172.21.15.94:0/3710250412 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7fb33800fa90 con 0x55f19aa5e5b0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3380105b0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb33ea00640 1 -- 172.21.15.94:0/3710250412 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb33803a050 con 0x55f19aa5e5b0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb35ab98740 10 client.25831 fetch_fsmap learned FSMap version 37 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.644+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f19aa70720 con 0x55f19aa5e5b0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb33ea00640 1 -- 172.21.15.94:0/3710250412 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3380105b0 con 0x55f19aa5e5b0 2024-06-03T01:16:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb33ea00640 1 -- 172.21.15.94:0/3710250412 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb338010810 con 0x55f19aa5e5b0 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb35ab98740 10 client.25831 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb35ab98740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb35ab98740 20 client.25831 populate_metadata read hostname 'smithi094' 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aae3150 0x55f19aae5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb35ab98740 4 client.25831 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f19a9467a0 con 0x55f19aae3150 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.645+0000 7fb355c00640 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aae3150 0x55f19aae5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.646+0000 7fb355c00640 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aae3150 0x55f19aae5540 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af70e797fd287b16 server_cookie=11f8b58776c923d in_seq=0 out_seq=0 2024-06-03T01:16:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.646+0000 7fb33ea00640 10 client.25831 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.680+0000 7fb33ea00640 1 -- 172.21.15.94:0/3710250412 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f19a9467a0 con 0x55f19aae3150 2024-06-03T01:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.680+0000 7fb33ea00640 10 client.25831 handle_command_reply: tid=0 2024-06-03T01:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.680+0000 7fb35ab98740 1 client.25831 shutdown 2024-06-03T01:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.680+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 msgr2=0x55f19aa68ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.680+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa68ce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55f19aa5a140 tx=0x7fb338003820 comp rx=0 tx=0).stop 2024-06-03T01:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 shutdown_connections 2024-06-03T01:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55f19aae3150 0x55f19aae5540 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f19aa67dc0 0x55f19aa6efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f19aa5e5b0 0x55f19aa68ce0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 --2- 172.21.15.94:0/3710250412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f19aa5d040 0x55f19aa687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 >> 172.21.15.94:0/3710250412 conn(0x55f19aa25f80 msgr2=0x55f19aa72e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb33ea00640 0 client.25831 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 shutdown_connections 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 1 -- 172.21.15.94:0/3710250412 wait complete. 2024-06-03T01:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.681+0000 7fb35ab98740 20 client.25831 trim_cache size 0 max 16384 2024-06-03T01:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb3340a2da0 msgr2=0x7fb3340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb3340a2da0 0x7fb3340a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55f19aa5fd10 tx=0x7fb3400f3610 comp rx=0 tx=0).stop 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb350103e10 msgr2=0x7fb35019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb350103e10 0x7fb35019a430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb340006010 tx=0x7fb340004040 comp rx=0 tx=0).stop 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 shutdown_connections 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb3340a2da0 0x7fb3340a5260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb35019ecf0 0x7fb35019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb350108ee0 0x7fb35019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 --2- 172.21.15.94:0/1368349455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb350103e10 0x7fb35019a430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 >> 172.21.15.94:0/1368349455 conn(0x7fb3500ff6b0 msgr2=0x7fb350101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 shutdown_connections 2024-06-03T01:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:16:47.682+0000 7fb357c00640 1 -- 172.21.15.94:0/1368349455 wait complete. 2024-06-03T01:16:48.093 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146182 2.20 MB/sec execute 175 sec latency 1942.610 ms 2024-06-03T01:16:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:48 smithi179 ceph-mon[37041]: pgmap v196: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 907 KiB/s wr, 121 op/s 2024-06-03T01:16:48.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:48 smithi107 ceph-mon[36622]: pgmap v196: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 907 KiB/s wr, 121 op/s 2024-06-03T01:16:48.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:48 smithi094 ceph-mon[32119]: pgmap v196: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 907 KiB/s wr, 121 op/s 2024-06-03T01:16:49.092 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146182 2.18 MB/sec execute 176 sec latency 2942.779 ms 2024-06-03T01:16:50.093 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146182 2.17 MB/sec execute 177 sec latency 3942.929 ms 2024-06-03T01:16:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:50 smithi107 ceph-mon[36622]: pgmap v197: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 595 KiB/s wr, 66 op/s 2024-06-03T01:16:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:50 smithi094 ceph-mon[32119]: pgmap v197: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 595 KiB/s wr, 66 op/s 2024-06-03T01:16:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:50 smithi179 ceph-mon[37041]: pgmap v197: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 595 KiB/s wr, 66 op/s 2024-06-03T01:16:51.094 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146201 2.16 MB/sec execute 178 sec latency 4901.002 ms 2024-06-03T01:16:52.094 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146250 2.15 MB/sec execute 179 sec latency 939.456 ms 2024-06-03T01:16:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:52 smithi107 ceph-mon[36622]: pgmap v198: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 595 KiB/s wr, 66 op/s 2024-06-03T01:16:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:52 smithi094 ceph-mon[32119]: pgmap v198: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 595 KiB/s wr, 66 op/s 2024-06-03T01:16:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:52 smithi179 ceph-mon[37041]: pgmap v198: 129 pgs: 129 active+clean; 62 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 595 KiB/s wr, 66 op/s 2024-06-03T01:16:53.094 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146250 2.14 MB/sec execute 180 sec latency 1939.600 ms 2024-06-03T01:16:54.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146250 2.12 MB/sec execute 181 sec latency 2939.720 ms 2024-06-03T01:16:54.406 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:54 smithi179 ceph-mon[37041]: pgmap v199: 129 pgs: 129 active+clean; 62 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 998 KiB/s wr, 98 op/s 2024-06-03T01:16:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:54 smithi107 ceph-mon[36622]: pgmap v199: 129 pgs: 129 active+clean; 62 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 998 KiB/s wr, 98 op/s 2024-06-03T01:16:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:54 smithi094 ceph-mon[32119]: pgmap v199: 129 pgs: 129 active+clean; 62 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 998 KiB/s wr, 98 op/s 2024-06-03T01:16:55.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146250 2.11 MB/sec execute 182 sec latency 3939.955 ms 2024-06-03T01:16:56.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 146269 2.10 MB/sec execute 183 sec latency 4900.664 ms 2024-06-03T01:16:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:56 smithi107 ceph-mon[36622]: pgmap v200: 129 pgs: 129 active+clean; 62 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 677 KiB/s wr, 74 op/s 2024-06-03T01:16:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:56 smithi094 ceph-mon[32119]: pgmap v200: 129 pgs: 129 active+clean; 62 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 677 KiB/s wr, 74 op/s 2024-06-03T01:16:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:56 smithi179 ceph-mon[37041]: pgmap v200: 129 pgs: 129 active+clean; 62 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 677 KiB/s wr, 74 op/s 2024-06-03T01:16:57.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 147878 2.13 MB/sec execute 184 sec latency 18.244 ms 2024-06-03T01:16:58.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 149213 2.18 MB/sec execute 185 sec latency 18.966 ms 2024-06-03T01:16:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:16:58 smithi107 ceph-mon[36622]: pgmap v201: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1023 KiB/s wr, 103 op/s 2024-06-03T01:16:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:16:58 smithi094 ceph-mon[32119]: pgmap v201: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1023 KiB/s wr, 103 op/s 2024-06-03T01:16:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:16:58 smithi179 ceph-mon[37041]: pgmap v201: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1023 KiB/s wr, 103 op/s 2024-06-03T01:16:59.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 149723 2.17 MB/sec execute 186 sec latency 141.496 ms 2024-06-03T01:17:00.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 149723 2.16 MB/sec execute 187 sec latency 1141.659 ms 2024-06-03T01:17:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:00 smithi107 ceph-mon[36622]: pgmap v202: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 748 KiB/s wr, 60 op/s 2024-06-03T01:17:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:00 smithi094 ceph-mon[32119]: pgmap v202: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 748 KiB/s wr, 60 op/s 2024-06-03T01:17:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:00 smithi179 ceph-mon[37041]: pgmap v202: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 748 KiB/s wr, 60 op/s 2024-06-03T01:17:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 -- 172.21.15.94:0/120374054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41029c0 msgr2=0x7f48c4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 --2- 172.21.15.94:0/120374054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41029c0 0x7f48c4102da0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009900 tx=0x7f48b402a480 comp rx=0 tx=0).stop 2024-06-03T01:17:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 -- 172.21.15.94:0/120374054 shutdown_connections 2024-06-03T01:17:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 --2- 172.21.15.94:0/120374054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48c410bd90 0x7f48c410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 --2- 172.21.15.94:0/120374054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48c41032e0 0x7f48c410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 --2- 172.21.15.94:0/120374054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41029c0 0x7f48c4102da0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 -- 172.21.15.94:0/120374054 >> 172.21.15.94:0/120374054 conn(0x7f48c40fc1c0 msgr2=0x7f48c40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 -- 172.21.15.94:0/120374054 shutdown_connections 2024-06-03T01:17:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.845+0000 7f48c9e00640 1 -- 172.21.15.94:0/120374054 wait complete. 2024-06-03T01:17:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 Processor -- start 2024-06-03T01:17:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 -- start start 2024-06-03T01:17:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48c41029c0 0x7f48c419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41032e0 0x7f48c419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48c410bd90 0x7f48c419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4111bc0 con 0x7f48c41029c0 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4111a40 con 0x7f48c410bd90 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.846+0000 7f48c9e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4111d40 con 0x7f48c41032e0 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41032e0 0x7f48c419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41032e0 0x7f48c419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38474/0 (socket says 172.21.15.94:38474) 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 -- 172.21.15.94:0/2744737122 learned_addr learned my addr 172.21.15.94:0/2744737122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48c8a00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48c41029c0 0x7f48c419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48c9400640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48c410bd90 0x7f48c419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 -- 172.21.15.94:0/2744737122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48c410bd90 msgr2=0x7f48c419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48c410bd90 0x7f48c419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 -- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48c41029c0 msgr2=0x7f48c419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48c41029c0 0x7f48c419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 -- 172.21.15.94:0/2744737122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c419f290 con 0x7f48c41032e0 2024-06-03T01:17:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.847+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41032e0 0x7f48c419a7e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48b000d950 tx=0x7f48b000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.848+0000 7f48c2000640 1 -- 172.21.15.94:0/2744737122 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0017030 con 0x7f48c41032e0 2024-06-03T01:17:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.848+0000 7f48c2000640 1 -- 172.21.15.94:0/2744737122 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b0004020 con 0x7f48c41032e0 2024-06-03T01:17:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.848+0000 7f48c2000640 1 -- 172.21.15.94:0/2744737122 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00047a0 con 0x7f48c41032e0 2024-06-03T01:17:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.848+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c419f580 con 0x7f48c41032e0 2024-06-03T01:17:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.848+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c41a6e10 con 0x7f48c41032e0 2024-06-03T01:17:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 Processor -- start 2024-06-03T01:17:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 -- start start 2024-06-03T01:17:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab709cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab70a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab717370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624ab66e420 con 0x5624ab70b770 2024-06-03T01:17:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5624ab615c50 con 0x5624ab714f80 2024-06-03T01:17:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5624ab66a3e0 con 0x5624ab70a200 2024-06-03T01:17:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.852+0000 7f48c2000640 1 -- 172.21.15.94:0/2744737122 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f48b00041d0 con 0x7f48c41032e0 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab709cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab70a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab709cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47642/0 (socket says 172.21.15.94:47642) 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 -- 172.21.15.94:0/2612738969 learned_addr learned my addr 172.21.15.94:0/2612738969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab717370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 -- 172.21.15.94:0/2612738969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 msgr2=0x5624ab70a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab70a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 -- 172.21.15.94:0/2612738969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 msgr2=0x5624ab717370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab717370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.853+0000 7f48c9400640 1 -- 172.21.15.94:0/2612738969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ab382210 con 0x5624ab70b770 2024-06-03T01:17:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.854+0000 7f48c9400640 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab709cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009f50 tx=0x7f48bc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680116961ee0fc49 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.854+0000 7f48b9c00640 1 -- 172.21.15.94:0/2612738969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc004d20 con 0x5624ab70b770 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.854+0000 7f48b9c00640 1 -- 172.21.15.94:0/2612738969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc004050 con 0x5624ab70b770 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.854+0000 7f48b9c00640 1 -- 172.21.15.94:0/2612738969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc005260 con 0x5624ab70b770 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48c2000640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f48a80a2d70 0x7f48a80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2612738969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 msgr2=0x5624ab709cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab709cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009f50 tx=0x7f48bc0118d0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48c2000640 1 -- 172.21.15.94:0/2744737122 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f48b00cf060 con 0x7f48c41032e0 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48c8a00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f48a80a2d70 0x7f48a80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2612738969 shutdown_connections 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab717370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab70a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2612738969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab709cc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.855+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2612738969 >> 172.21.15.94:0/2612738969 conn(0x5624ab6d2f80 msgr2=0x5624ab7067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.856+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2612738969 shutdown_connections 2024-06-03T01:17:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.856+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2612738969 wait complete. 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.856+0000 7f48c8a00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f48a80a2d70 0x7f48a80a5230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48b4000c00 tx=0x7f48b4005b50 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.856+0000 7f48ce3e0740 1 -- start start 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 1 Processor -- start 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab711f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab7124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab712a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5624ab717d50 con 0x5624ab70b770 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5624ab6290b0 con 0x5624ab714f80 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5624ab717ed0 con 0x5624ab70a200 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48ce3e0740 10 client.0 _ll_register_callbacks cb 0x5624ab614f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48c8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab711f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48c8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab711f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38482/0 (socket says 172.21.15.94:38482) 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.857+0000 7f48c8a00640 1 -- 172.21.15.94:0/2717805633 learned_addr learned my addr 172.21.15.94:0/2717805633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab7124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c8a00640 1 -- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 msgr2=0x5624ab712a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c9400640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab712a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c8a00640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab712a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c8a00640 1 -- 172.21.15.94:0/2717805633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 msgr2=0x5624ab7124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c8a00640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab7124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c8a00640 1 -- 172.21.15.94:0/2717805633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ab44e620 con 0x5624ab70a200 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab7124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c9400640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab712a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48c8a00640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab711f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009900 tx=0x7f48b4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a24c1e4ddd99e42 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.858+0000 7f48ae000640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ae000640 1 -- 172.21.15.94:0/2717805633 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b404e070 con 0x5624ab70a200 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48bc004050 con 0x5624ab70a200 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ae000640 1 -- 172.21.15.94:0/2717805633 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f48b4004c50 con 0x5624ab70a200 2024-06-03T01:17:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ae000640 1 -- 172.21.15.94:0/2717805633 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f48b405a050 con 0x5624ab70a200 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ce3e0740 10 client.25843 fetch_fsmap learned FSMap version 37 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5624ab724110 con 0x5624ab70a200 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ae000640 1 -- 172.21.15.94:0/2717805633 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48bc004050 con 0x5624ab70a200 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.859+0000 7f48ae000640 1 -- 172.21.15.94:0/2717805633 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f48b4003790 con 0x5624ab70a200 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48ce3e0740 10 client.25843 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48ce3e0740 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48ce3e0740 20 client.25843 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab725580 0x5624ab727970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48ce3e0740 4 client.25843 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624ab67bb50 con 0x5624ab725580 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab725580 0x5624ab727970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48bbe00640 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab725580 0x5624ab727970 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acb3f690033cef5 server_cookie=fdb0b0a8bf7d6236 in_seq=0 out_seq=0 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.860+0000 7f48ae000640 10 client.25843 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.861+0000 7f48ae000640 1 -- 172.21.15.94:0/2717805633 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5624ab67bb50 con 0x5624ab725580 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.861+0000 7f48ae000640 10 client.25843 handle_command_reply: tid=0 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.861+0000 7f48ce3e0740 1 client.25843 shutdown 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.861+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 msgr2=0x5624ab711f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.861+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab711f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009900 tx=0x7f48b4009260 comp rx=0 tx=0).stop 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 shutdown_connections 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab725580 0x5624ab727970 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab712a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70b770 0x5624ab7124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/2717805633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70a200 0x5624ab711f90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 >> 172.21.15.94:0/2717805633 conn(0x5624ab6d2f80 msgr2=0x5624ab70c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ae000640 0 client.25843 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 shutdown_connections 2024-06-03T01:17:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 1 -- 172.21.15.94:0/2717805633 wait complete. 2024-06-03T01:17:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.862+0000 7f48ce3e0740 20 client.25843 trim_cache size 0 max 16384 2024-06-03T01:17:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.879+0000 7f48ce3e0740 1 Processor -- start 2024-06-03T01:17:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48ce3e0740 1 -- start start 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab72d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab70b770 0x5624ab72d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab714f80 0x5624ab70bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b0d60 con 0x5624ab70a200 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b28a0 con 0x5624ab70b770 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b1430 con 0x5624ab714f80 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab70b770 0x5624ab72d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48c8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab72d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab70b770 0x5624ab72d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45248/0 (socket says 172.21.15.94:45248) 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48bbe00640 1 -- 172.21.15.94:0/1986708620 learned_addr learned my addr 172.21.15.94:0/1986708620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.880+0000 7f48c9400640 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab714f80 0x5624ab70bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48bbe00640 1 -- 172.21.15.94:0/1986708620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab714f80 msgr2=0x5624ab70bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48bbe00640 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab714f80 0x5624ab70bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48bbe00640 1 -- 172.21.15.94:0/1986708620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 msgr2=0x5624ab72d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48bbe00640 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab72d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48bbe00640 1 -- 172.21.15.94:0/1986708620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ab70b380 con 0x5624ab70b770 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48c8a00640 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab72d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:17:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48bbe00640 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab70b770 0x5624ab72d7c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48b0097680 tx=0x7f48b00d15b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64256ab1186b1922 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.881+0000 7f48b9200640 1 -- 172.21.15.94:0/1986708620 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00b9e60 con 0x5624ab70b770 2024-06-03T01:17:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48b9200640 1 -- 172.21.15.94:0/1986708620 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b009b920 con 0x5624ab70b770 2024-06-03T01:17:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48b9200640 1 -- 172.21.15.94:0/1986708620 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00a3e90 con 0x5624ab70b770 2024-06-03T01:17:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1986708620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab70b770 msgr2=0x5624ab72d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab70b770 0x5624ab72d7c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48b0097680 tx=0x7f48b00d15b0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1986708620 shutdown_connections 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab714f80 0x5624ab70bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab70b770 0x5624ab72d7c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1986708620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab72d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.882+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1986708620 >> 172.21.15.94:0/1986708620 conn(0x5624ab6d2f80 msgr2=0x5624ab70d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.883+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1986708620 shutdown_connections 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.883+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1986708620 wait complete. 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.883+0000 7f48ce3e0740 1 -- start start 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.883+0000 7f48ce3e0740 1 Processor -- start 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab7188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70b770 0x5624ab718e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48ce3e0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab788280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b21d0 con 0x5624ab70a200 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41b1b00 con 0x5624ab714f80 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48ce3e0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f48c41aed40 con 0x5624ab70b770 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48ce3e0740 10 client.0 _ll_register_callbacks cb 0x7f48b40041b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70b770 0x5624ab718e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48c9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab788280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70b770 0x5624ab718e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38514/0 (socket says 172.21.15.94:38514) 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48c9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab788280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:45260/0 (socket says 172.21.15.94:45260) 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 -- 172.21.15.94:0/1487499030 learned_addr learned my addr 172.21.15.94:0/1487499030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48c8a00640 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab7188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 -- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 msgr2=0x5624ab788280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab788280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 -- 172.21.15.94:0/1487499030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 msgr2=0x5624ab7188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab7188d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48bbe00640 1 -- 172.21.15.94:0/1487499030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ab788870 con 0x5624ab70b770 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.884+0000 7f48c9400640 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab788280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48bbe00640 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70b770 0x5624ab718e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009610 tx=0x7f48b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0502783db9d48f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48ae000640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48ae000640 1 -- 172.21.15.94:0/1487499030 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b00b9e60 con 0x5624ab70b770 2024-06-03T01:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48ae000640 1 -- 172.21.15.94:0/1487499030 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9387+0+0 (secure 0 0 0) 0x7f48b00cc6d0 con 0x5624ab70b770 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b009b920 con 0x5624ab70b770 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48ae000640 1 -- 172.21.15.94:0/1487499030 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f48b009ad60 con 0x5624ab70b770 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48ce3e0740 10 client.25849 fetch_fsmap learned FSMap version 37 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.885+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5624ab78fc00 con 0x5624ab70b770 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.886+0000 7f48ae000640 1 -- 172.21.15.94:0/1487499030 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48b009b920 con 0x5624ab70b770 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.886+0000 7f48ae000640 1 -- 172.21.15.94:0/1487499030 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f48b009f180 con 0x5624ab70b770 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.887+0000 7f48ce3e0740 10 client.25849 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.887+0000 7f48ce3e0740 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.887+0000 7f48ce3e0740 20 client.25849 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.887+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab790220 0x5624ab792610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.887+0000 7f48ce3e0740 4 client.25849 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.887+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624ab675ea0 con 0x5624ab790220 2024-06-03T01:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.888+0000 7f48c8a00640 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab790220 0x5624ab792610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.888+0000 7f48c8a00640 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab790220 0x5624ab792610 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c604525f738daee server_cookie=7de90f9b285cded9 in_seq=0 out_seq=0 2024-06-03T01:17:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.888+0000 7f48ae000640 10 client.25849 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.918+0000 7f48ae000640 1 -- 172.21.15.94:0/1487499030 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5624ab675ea0 con 0x5624ab790220 2024-06-03T01:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.918+0000 7f48ae000640 10 client.25849 handle_command_reply: tid=0 2024-06-03T01:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.918+0000 7f48ce3e0740 1 client.25849 shutdown 2024-06-03T01:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70b770 msgr2=0x5624ab718e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70b770 0x5624ab718e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009610 tx=0x7f48b0000bc0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 shutdown_connections 2024-06-03T01:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5624ab790220 0x5624ab792610 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624ab714f80 0x5624ab788280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5624ab70b770 0x5624ab718e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 --2- 172.21.15.94:0/1487499030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5624ab70a200 0x5624ab7188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 >> 172.21.15.94:0/1487499030 conn(0x5624ab6d2f80 msgr2=0x5624ab711f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ae000640 0 client.25849 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 shutdown_connections 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.919+0000 7f48ce3e0740 1 -- 172.21.15.94:0/1487499030 wait complete. 2024-06-03T01:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.920+0000 7f48ce3e0740 20 client.25849 trim_cache size 0 max 16384 2024-06-03T01:17:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:17:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:17:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:17:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f48a80a2d70 msgr2=0x7f48a80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f48a80a2d70 0x7f48a80a5230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48b4000c00 tx=0x7f48b4005b50 comp rx=0 tx=0).stop 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41032e0 msgr2=0x7f48c419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41032e0 0x7f48c419a7e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48b000d950 tx=0x7f48b000de10 comp rx=0 tx=0).stop 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 shutdown_connections 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f48a80a2d70 0x7f48a80a5230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48c410bd90 0x7f48c419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48c41032e0 0x7f48c419a7e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 --2- 172.21.15.94:0/2744737122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f48c41029c0 0x7f48c419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 >> 172.21.15.94:0/2744737122 conn(0x7f48c40fc1c0 msgr2=0x7f48c4100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 shutdown_connections 2024-06-03T01:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:00.921+0000 7f48c9e00640 1 -- 172.21.15.94:0/2744737122 wait complete. 2024-06-03T01:17:01.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 150008 2.15 MB/sec execute 188 sec latency 1806.783 ms 2024-06-03T01:17:02.095 INFO:tasks.workunit.client.0.smithi094.stdout: 1 150034 2.14 MB/sec execute 189 sec latency 96.663 ms 2024-06-03T01:17:02.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:02 smithi107 ceph-mon[36622]: pgmap v203: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 748 KiB/s wr, 60 op/s 2024-06-03T01:17:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:02 smithi094 ceph-mon[32119]: pgmap v203: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 748 KiB/s wr, 60 op/s 2024-06-03T01:17:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:02 smithi179 ceph-mon[37041]: pgmap v203: 129 pgs: 129 active+clean; 62 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 748 KiB/s wr, 60 op/s 2024-06-03T01:17:03.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 151550 2.16 MB/sec execute 190 sec latency 97.270 ms 2024-06-03T01:17:04.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 152796 2.21 MB/sec execute 191 sec latency 21.222 ms 2024-06-03T01:17:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:04 smithi107 ceph-mon[36622]: pgmap v204: 129 pgs: 129 active+clean; 60 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-03T01:17:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:04 smithi094 ceph-mon[32119]: pgmap v204: 129 pgs: 129 active+clean; 60 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-03T01:17:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:04 smithi179 ceph-mon[37041]: pgmap v204: 129 pgs: 129 active+clean; 60 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-03T01:17:05.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 153341 2.20 MB/sec execute 192 sec latency 36.560 ms 2024-06-03T01:17:05.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:05 smithi107 ceph-mon[36622]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T01:17:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:05 smithi094 ceph-mon[32119]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T01:17:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:05 smithi179 ceph-mon[37041]: osdmap e102: 12 total, 12 up, 12 in 2024-06-03T01:17:06.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 153373 2.19 MB/sec execute 193 sec latency 980.241 ms 2024-06-03T01:17:06.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:06 smithi107 ceph-mon[36622]: pgmap v206: 129 pgs: 129 active+clean; 60 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 892 KiB/s wr, 131 op/s 2024-06-03T01:17:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:06 smithi094 ceph-mon[32119]: pgmap v206: 129 pgs: 129 active+clean; 60 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 892 KiB/s wr, 131 op/s 2024-06-03T01:17:06.604 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:06 smithi179 ceph-mon[37041]: pgmap v206: 129 pgs: 129 active+clean; 60 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 892 KiB/s wr, 131 op/s 2024-06-03T01:17:07.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 154957 2.21 MB/sec execute 194 sec latency 16.927 ms 2024-06-03T01:17:08.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156224 2.26 MB/sec execute 195 sec latency 16.892 ms 2024-06-03T01:17:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:08 smithi107 ceph-mon[36622]: pgmap v207: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-06-03T01:17:08.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:08 smithi094 ceph-mon[32119]: pgmap v207: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-06-03T01:17:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:08 smithi179 ceph-mon[37041]: pgmap v207: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-06-03T01:17:09.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156759 2.25 MB/sec execute 196 sec latency 42.051 ms 2024-06-03T01:17:09.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:09 smithi094 ceph-mon[32119]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T01:17:09.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:09 smithi179 ceph-mon[37041]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T01:17:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:09 smithi107 ceph-mon[36622]: osdmap e103: 12 total, 12 up, 12 in 2024-06-03T01:17:10.096 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156759 2.24 MB/sec execute 197 sec latency 1042.193 ms 2024-06-03T01:17:10.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:10 smithi094 ceph-mon[32119]: pgmap v209: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-06-03T01:17:10.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:10 smithi179 ceph-mon[37041]: pgmap v209: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-06-03T01:17:11.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:10 smithi107 ceph-mon[36622]: pgmap v209: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-06-03T01:17:11.097 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156778 2.23 MB/sec execute 198 sec latency 1998.361 ms 2024-06-03T01:17:12.097 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156815 2.22 MB/sec execute 199 sec latency 950.555 ms 2024-06-03T01:17:12.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:12 smithi094 ceph-mon[32119]: pgmap v210: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1002 KiB/s wr, 106 op/s 2024-06-03T01:17:12.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:12 smithi179 ceph-mon[37041]: pgmap v210: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1002 KiB/s wr, 106 op/s 2024-06-03T01:17:13.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:12 smithi107 ceph-mon[36622]: pgmap v210: 129 pgs: 129 active+clean; 64 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1002 KiB/s wr, 106 op/s 2024-06-03T01:17:13.097 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156815 2.21 MB/sec execute 200 sec latency 1950.687 ms 2024-06-03T01:17:14.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:13 smithi107 ceph-mon[36622]: pgmap v211: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 -- 172.21.15.94:0/396521835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01019b0 msgr2=0x7facc0101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 --2- 172.21.15.94:0/396521835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01019b0 0x7facc0101e30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac02a3f0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 -- 172.21.15.94:0/396521835 shutdown_connections 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 --2- 172.21.15.94:0/396521835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc0102370 0x7facc010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 --2- 172.21.15.94:0/396521835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01019b0 0x7facc0101e30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 --2- 172.21.15.94:0/396521835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facc01079c0 0x7facc0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 -- 172.21.15.94:0/396521835 >> 172.21.15.94:0/396521835 conn(0x7facc0078e30 msgr2=0x7facc0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 -- 172.21.15.94:0/396521835 shutdown_connections 2024-06-03T01:17:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.063+0000 7facc5c00640 1 -- 172.21.15.94:0/396521835 wait complete. 2024-06-03T01:17:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.064+0000 7facc5c00640 1 Processor -- start 2024-06-03T01:17:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facc5c00640 1 -- start start 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facc5c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc01019b0 0x7facc019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facc5c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facc0102370 0x7facc019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facc5c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01079c0 0x7facc019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facc5c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7facc0111e20 con 0x7facc01019b0 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facc5c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7facc0111ca0 con 0x7facc01079c0 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facc5c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7facc0111fa0 con 0x7facc0102370 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facbea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc01019b0 0x7facc019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facbea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc01019b0 0x7facc019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59190/0 (socket says 172.21.15.94:59190) 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facbe000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facc0102370 0x7facc019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facbea00640 1 -- 172.21.15.94:0/880362151 learned_addr learned my addr 172.21.15.94:0/880362151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.065+0000 7facbf400640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01079c0 0x7facc019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facbea00640 1 -- 172.21.15.94:0/880362151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facc0102370 msgr2=0x7facc019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facbea00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facc0102370 0x7facc019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facbea00640 1 -- 172.21.15.94:0/880362151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01079c0 msgr2=0x7facc019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facbea00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01079c0 0x7facc019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facbea00640 1 -- 172.21.15.94:0/880362151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc019f430 con 0x7facc01019b0 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facbea00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc01019b0 0x7facc019a440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7facb000d770 tx=0x7facb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facb7400640 1 -- 172.21.15.94:0/880362151 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0017070 con 0x7facc01019b0 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facb7400640 1 -- 172.21.15.94:0/880362151 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb0005500 con 0x7facc01019b0 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facb7400640 1 -- 172.21.15.94:0/880362151 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0004e00 con 0x7facc01019b0 2024-06-03T01:17:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc019f720 con 0x7facc01019b0 2024-06-03T01:17:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.066+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01a6fb0 con 0x7facc01019b0 2024-06-03T01:17:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 Processor -- start 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facb7400640 1 -- 172.21.15.94:0/880362151 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7facb0004fa0 con 0x7facc01019b0 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 -- start start 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559eeac86420 con 0x559eead235b0 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559eeac2dc50 con 0x559eead2cdc0 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facc8b83740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559eeac823e0 con 0x559eead22040 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facbf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facbf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead21b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59198/0 (socket says 172.21.15.94:59198) 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facbf400640 1 -- 172.21.15.94:0/1415733006 learned_addr learned my addr 172.21.15.94:0/1415733006 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facbea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.070+0000 7facbe000640 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facbf400640 1 -- 172.21.15.94:0/1415733006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 msgr2=0x559eead224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facbf400640 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facbf400640 1 -- 172.21.15.94:0/1415733006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 msgr2=0x559eead2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facbf400640 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facbf400640 1 -- 172.21.15.94:0/1415733006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eea99a210 con 0x559eead235b0 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facbe000640 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facbf400640 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead21b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fc0 tx=0x7facb80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2327cb39867fd27b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facb7400640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faca40a2da0 0x7faca40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.071+0000 7facabe00640 1 -- 172.21.15.94:0/1415733006 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8004d20 con 0x559eead235b0 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facabe00640 1 -- 172.21.15.94:0/1415733006 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8004050 con 0x559eead235b0 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facabe00640 1 -- 172.21.15.94:0/1415733006 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8005220 con 0x559eead235b0 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facb7400640 1 -- 172.21.15.94:0/880362151 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7facb00cf190 con 0x7facc01019b0 2024-06-03T01:17:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facbea00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faca40a2da0 0x7faca40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 -- 172.21.15.94:0/1415733006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 msgr2=0x559eead21b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facbea00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faca40a2da0 0x7faca40a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x559eead24d10 tx=0x7facb00bb040 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead21b00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fc0 tx=0x7facb80118d0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 -- 172.21.15.94:0/1415733006 shutdown_connections 2024-06-03T01:17:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 --2- 172.21.15.94:0/1415733006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead21b00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 -- 172.21.15.94:0/1415733006 >> 172.21.15.94:0/1415733006 conn(0x559eeaceaf80 msgr2=0x559eead1e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.072+0000 7facc8b83740 1 -- 172.21.15.94:0/1415733006 shutdown_connections 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.073+0000 7facc8b83740 1 -- 172.21.15.94:0/1415733006 wait complete. 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.073+0000 7facc8b83740 1 -- start start 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.073+0000 7facc8b83740 1 Processor -- start 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facc8b83740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead29d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facc8b83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facc8b83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facc8b83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559eead2fb90 con 0x559eead235b0 2024-06-03T01:17:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facc8b83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559eeac410b0 con 0x559eead2cdc0 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facc8b83740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559eead2fd10 con 0x559eead22040 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facc8b83740 10 client.0 _ll_register_callbacks cb 0x559eeac2cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59210/0 (socket says 172.21.15.94:59210) 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 -- 172.21.15.94:0/881792898 learned_addr learned my addr 172.21.15.94:0/881792898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbf400640 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbf400640 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 -- 172.21.15.94:0/881792898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 msgr2=0x559eead29d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead29d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 -- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 msgr2=0x559eead2a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.074+0000 7facbe000640 1 -- 172.21.15.94:0/881792898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eeaa66620 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7facbe000640 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a2d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f051390907e91e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7faca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7faca8c00640 1 -- 172.21.15.94:0/881792898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac035040 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7faca8c00640 1 -- 172.21.15.94:0/881792898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7facac005450 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb8004050 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7faca8c00640 1 -- 172.21.15.94:0/881792898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7facac04f050 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7facc8b83740 10 client.15726 fetch_fsmap learned FSMap version 37 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.075+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559eead3bec0 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7faca8c00640 1 -- 172.21.15.94:0/881792898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8004050 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7faca8c00640 1 -- 172.21.15.94:0/881792898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7facac0038f0 con 0x559eead235b0 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7facc8b83740 10 client.15726 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7facc8b83740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7facc8b83740 20 client.15726 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7facc8b83740 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eead3d3d0 0x559eead3f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7facc8b83740 4 client.15726 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:17:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eeac93b50 con 0x559eead3d3d0 2024-06-03T01:17:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.076+0000 7facbf400640 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eead3d3d0 0x559eead3f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.077+0000 7facbf400640 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eead3d3d0 0x559eead3f7c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6922f4dec76a57e7 server_cookie=5a21dddd6253dd94 in_seq=0 out_seq=0 2024-06-03T01:17:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.077+0000 7faca8c00640 10 client.15726 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7faca8c00640 1 -- 172.21.15.94:0/881792898 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559eeac93b50 con 0x559eead3d3d0 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7faca8c00640 10 client.15726 handle_command_reply: tid=0 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 client.15726 shutdown 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 msgr2=0x559eead2a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a2d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac0043b0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 shutdown_connections 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eead3d3d0 0x559eead3f7c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead2a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a2d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 --2- 172.21.15.94:0/881792898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead29d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7faca8c00640 0 client.15726 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 >> 172.21.15.94:0/881792898 conn(0x559eeaceaf80 msgr2=0x559eead24430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 shutdown_connections 2024-06-03T01:17:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 1 -- 172.21.15.94:0/881792898 wait complete. 2024-06-03T01:17:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.078+0000 7facc8b83740 20 client.15726 trim_cache size 0 max 16384 2024-06-03T01:17:14.097 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156815 2.20 MB/sec execute 201 sec latency 2950.783 ms 2024-06-03T01:17:14.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:13 smithi094 ceph-mon[32119]: pgmap v211: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-03T01:17:14.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:13 smithi179 ceph-mon[37041]: pgmap v211: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-03T01:17:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 Processor -- start 2024-06-03T01:17:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 -- start start 2024-06-03T01:17:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead1e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7facc01b1000 con 0x559eead235b0 2024-06-03T01:17:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7facc01b2b40 con 0x559eead2cdc0 2024-06-03T01:17:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.108+0000 7facc8b83740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7facc01b16d0 con 0x559eead22040 2024-06-03T01:17:14.109 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbf400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead1e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbf400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead1e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:38668/0 (socket says 172.21.15.94:38668) 2024-06-03T01:17:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbf400640 1 -- 172.21.15.94:0/2152066758 learned_addr learned my addr 172.21.15.94:0/2152066758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbf400640 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbe000640 1 -- 172.21.15.94:0/2152066758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 msgr2=0x559eead1e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbe000640 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead1e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbe000640 1 -- 172.21.15.94:0/2152066758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 msgr2=0x559eead242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbe000640 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbe000640 1 -- 172.21.15.94:0/2152066758 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eead282e0 con 0x559eead235b0 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.109+0000 7facbf400640 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead1e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.110+0000 7facbe000640 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7facac003e30 tx=0x7facac005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab01841d3522cb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.110+0000 7facab400640 1 -- 172.21.15.94:0/2152066758 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac035030 con 0x559eead235b0 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.110+0000 7facab400640 1 -- 172.21.15.94:0/2152066758 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac041070 con 0x559eead235b0 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.110+0000 7facab400640 1 -- 172.21.15.94:0/2152066758 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac003970 con 0x559eead235b0 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 -- 172.21.15.94:0/2152066758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 msgr2=0x559eead2a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7facac003e30 tx=0x7facac005a40 comp rx=0 tx=0).stop 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 -- 172.21.15.94:0/2152066758 shutdown_connections 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead235b0 0x559eead2a630 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 --2- 172.21.15.94:0/2152066758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead22040 0x559eead1e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 -- 172.21.15.94:0/2152066758 >> 172.21.15.94:0/2152066758 conn(0x559eeaceaf80 msgr2=0x559eead23ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 -- 172.21.15.94:0/2152066758 shutdown_connections 2024-06-03T01:17:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 -- 172.21.15.94:0/2152066758 wait complete. 2024-06-03T01:17:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.111+0000 7facc8b83740 1 -- start start 2024-06-03T01:17:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 1 Processor -- start 2024-06-03T01:17:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead22040 0x559eead356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead235b0 0x559eead30a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead30fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7facc01b2470 con 0x559eead22040 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7facc01b1da0 con 0x559eead2cdc0 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7facc01af000 con 0x559eead235b0 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facc8b83740 10 client.0 _ll_register_callbacks cb 0x7facac005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facbf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead22040 0x559eead356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facbf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead22040 0x559eead356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59226/0 (socket says 172.21.15.94:59226) 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facbf400640 1 -- 172.21.15.94:0/1042251327 learned_addr learned my addr 172.21.15.94:0/1042251327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facbe000640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead235b0 0x559eead30a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.112+0000 7facbf400640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead30fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facbe000640 1 -- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 msgr2=0x559eead30fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facbe000640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead30fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facbe000640 1 -- 172.21.15.94:0/1042251327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead22040 msgr2=0x559eead356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facbe000640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead22040 0x559eead356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facbe000640 1 -- 172.21.15.94:0/1042251327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eead317c0 con 0x559eead235b0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facbf400640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead22040 0x559eead356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facbe000640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead235b0 0x559eead30a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x559eead227b0 tx=0x7facac02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63b03dae8083ce71 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7faca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7faca8c00640 1 -- 172.21.15.94:0/1042251327 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac035070 con 0x559eead235b0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facac041070 con 0x559eead235b0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.113+0000 7faca8c00640 1 -- 172.21.15.94:0/1042251327 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7facac005460 con 0x559eead235b0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7faca8c00640 1 -- 172.21.15.94:0/1042251327 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7facac066050 con 0x559eead235b0 2024-06-03T01:17:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 10 client.25867 fetch_fsmap learned FSMap version 37 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559eeada8060 con 0x559eead235b0 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7faca8c00640 1 -- 172.21.15.94:0/1042251327 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac041070 con 0x559eead235b0 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7faca8c00640 1 -- 172.21.15.94:0/1042251327 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7facac004b80 con 0x559eead235b0 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 10 client.25867 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 20 client.25867 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eeada8720 0x559eeadaab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 4 client.25867 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.114+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559eeac8dea0 con 0x559eeada8720 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.115+0000 7facbf400640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eeada8720 0x559eeadaab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.115+0000 7facbf400640 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eeada8720 0x559eeadaab10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2220e104b0cf41f server_cookie=16a678fc3c088760 in_seq=0 out_seq=0 2024-06-03T01:17:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.115+0000 7faca8c00640 10 client.25867 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.152+0000 7faca8c00640 1 -- 172.21.15.94:0/1042251327 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559eeac8dea0 con 0x559eeada8720 2024-06-03T01:17:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.152+0000 7faca8c00640 10 client.25867 handle_command_reply: tid=0 2024-06-03T01:17:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.152+0000 7facc8b83740 1 client.25867 shutdown 2024-06-03T01:17:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.152+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead235b0 msgr2=0x559eead30a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.152+0000 7facc8b83740 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead235b0 0x559eead30a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x559eead227b0 tx=0x7facac02a3f0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 shutdown_connections 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559eeada8720 0x559eeadaab10 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559eead2cdc0 0x559eead30fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559eead235b0 0x559eead30a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 --2- 172.21.15.94:0/1042251327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559eead22040 0x559eead356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7faca8c00640 0 client.25867 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 >> 172.21.15.94:0/1042251327 conn(0x559eeaceaf80 msgr2=0x559eead1d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 shutdown_connections 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 1 -- 172.21.15.94:0/1042251327 wait complete. 2024-06-03T01:17:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.153+0000 7facc8b83740 20 client.25867 trim_cache size 0 max 16384 2024-06-03T01:17:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:17:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:17:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faca40a2da0 msgr2=0x7faca40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faca40a2da0 0x7faca40a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x559eead24d10 tx=0x7facb00bb040 comp rx=0 tx=0).stop 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc01019b0 msgr2=0x7facc019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc01019b0 0x7facc019a440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7facb000d770 tx=0x7facb000dc30 comp rx=0 tx=0).stop 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 shutdown_connections 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faca40a2da0 0x7faca40a5260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc01079c0 0x7facc019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7facc0102370 0x7facc019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 --2- 172.21.15.94:0/880362151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7facc01019b0 0x7facc019a440 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 >> 172.21.15.94:0/880362151 conn(0x7facc0078e30 msgr2=0x7facc00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 shutdown_connections 2024-06-03T01:17:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:14.154+0000 7facc5c00640 1 -- 172.21.15.94:0/880362151 wait complete. 2024-06-03T01:17:15.097 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156815 2.19 MB/sec execute 202 sec latency 3950.919 ms 2024-06-03T01:17:16.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 156834 2.18 MB/sec execute 203 sec latency 4907.612 ms 2024-06-03T01:17:16.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:15 smithi107 ceph-mon[36622]: pgmap v212: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-06-03T01:17:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:15 smithi094 ceph-mon[32119]: pgmap v212: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-06-03T01:17:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:15 smithi179 ceph-mon[37041]: pgmap v212: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-06-03T01:17:17.097 INFO:tasks.workunit.client.0.smithi094.stdout: 1 158390 2.20 MB/sec execute 204 sec latency 18.991 ms 2024-06-03T01:17:18.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 159602 2.24 MB/sec execute 205 sec latency 20.067 ms 2024-06-03T01:17:18.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:17 smithi107 ceph-mon[36622]: pgmap v213: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 817 KiB/s wr, 101 op/s 2024-06-03T01:17:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:17 smithi094 ceph-mon[32119]: pgmap v213: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 817 KiB/s wr, 101 op/s 2024-06-03T01:17:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:17 smithi179 ceph-mon[37041]: pgmap v213: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 817 KiB/s wr, 101 op/s 2024-06-03T01:17:19.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160142 2.24 MB/sec execute 206 sec latency 19.446 ms 2024-06-03T01:17:20.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160362 2.23 MB/sec execute 207 sec latency 727.886 ms 2024-06-03T01:17:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:19 smithi107 ceph-mon[36622]: pgmap v214: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 792 KiB/s wr, 97 op/s 2024-06-03T01:17:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:19 smithi094 ceph-mon[32119]: pgmap v214: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 792 KiB/s wr, 97 op/s 2024-06-03T01:17:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:19 smithi179 ceph-mon[37041]: pgmap v214: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 792 KiB/s wr, 97 op/s 2024-06-03T01:17:21.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160384 2.22 MB/sec execute 208 sec latency 1683.753 ms 2024-06-03T01:17:22.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160430 2.21 MB/sec execute 209 sec latency 946.674 ms 2024-06-03T01:17:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:21 smithi107 ceph-mon[36622]: pgmap v215: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 681 KiB/s wr, 84 op/s 2024-06-03T01:17:22.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:21 smithi094 ceph-mon[32119]: pgmap v215: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 681 KiB/s wr, 84 op/s 2024-06-03T01:17:22.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:21 smithi179 ceph-mon[37041]: pgmap v215: 129 pgs: 129 active+clean; 63 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 681 KiB/s wr, 84 op/s 2024-06-03T01:17:23.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160430 2.20 MB/sec execute 210 sec latency 1946.820 ms 2024-06-03T01:17:24.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160430 2.18 MB/sec execute 211 sec latency 2946.960 ms 2024-06-03T01:17:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:23 smithi107 ceph-mon[36622]: pgmap v216: 129 pgs: 129 active+clean; 66 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 941 KiB/s wr, 122 op/s 2024-06-03T01:17:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:23 smithi094 ceph-mon[32119]: pgmap v216: 129 pgs: 129 active+clean; 66 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 941 KiB/s wr, 122 op/s 2024-06-03T01:17:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:23 smithi179 ceph-mon[37041]: pgmap v216: 129 pgs: 129 active+clean; 66 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 941 KiB/s wr, 122 op/s 2024-06-03T01:17:25.098 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160430 2.17 MB/sec execute 212 sec latency 3947.107 ms 2024-06-03T01:17:26.099 INFO:tasks.workunit.client.0.smithi094.stdout: 1 160449 2.16 MB/sec execute 213 sec latency 4905.014 ms 2024-06-03T01:17:26.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:26 smithi107 ceph-mon[36622]: pgmap v217: 129 pgs: 129 active+clean; 66 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 684 KiB/s wr, 82 op/s 2024-06-03T01:17:26.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:26 smithi094 ceph-mon[32119]: pgmap v217: 129 pgs: 129 active+clean; 66 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 684 KiB/s wr, 82 op/s 2024-06-03T01:17:26.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:26 smithi179 ceph-mon[37041]: pgmap v217: 129 pgs: 129 active+clean; 66 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 684 KiB/s wr, 82 op/s 2024-06-03T01:17:27.099 INFO:tasks.workunit.client.0.smithi094.stdout: 1 161381 2.18 MB/sec execute 214 sec latency 130.013 ms 2024-06-03T01:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.302+0000 7f7b59800640 1 -- 172.21.15.94:0/1901719308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b54102960 msgr2=0x7f7b54102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.302+0000 7f7b59800640 1 --2- 172.21.15.94:0/1901719308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b54102960 0x7f7b54102d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009be0 tx=0x7f7b4c02a750 comp rx=0 tx=0).stop 2024-06-03T01:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.302+0000 7f7b59800640 1 -- 172.21.15.94:0/1901719308 shutdown_connections 2024-06-03T01:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.302+0000 7f7b59800640 1 --2- 172.21.15.94:0/1901719308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b5410bd80 0x7f7b5410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.302+0000 7f7b59800640 1 --2- 172.21.15.94:0/1901719308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 0x7f7b5410b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.302+0000 7f7b59800640 1 --2- 172.21.15.94:0/1901719308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b54102960 0x7f7b54102d40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.302+0000 7f7b59800640 1 -- 172.21.15.94:0/1901719308 >> 172.21.15.94:0/1901719308 conn(0x7f7b540fc1c0 msgr2=0x7f7b540fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.303+0000 7f7b59800640 1 -- 172.21.15.94:0/1901719308 shutdown_connections 2024-06-03T01:17:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.303+0000 7f7b59800640 1 -- 172.21.15.94:0/1901719308 wait complete. 2024-06-03T01:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 Processor -- start 2024-06-03T01:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 -- start start 2024-06-03T01:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b54102960 0x7f7b541980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 0x7f7b541985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b5410bd80 0x7f7b5419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54111b30 con 0x7f7b5410bd80 2024-06-03T01:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541119b0 con 0x7f7b54103280 2024-06-03T01:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b59800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54111cb0 con 0x7f7b54102960 2024-06-03T01:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 0x7f7b541985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b53e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b54102960 0x7f7b541980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.304+0000 7f7b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 0x7f7b541985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56058/0 (socket says 172.21.15.94:56058) 2024-06-03T01:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b54102960 0x7f7b541980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:44084/0 (socket says 172.21.15.94:44084) 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53400640 1 -- 172.21.15.94:0/2483819048 learned_addr learned my addr 172.21.15.94:0/2483819048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b58e00640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b5410bd80 0x7f7b5419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53400640 1 -- 172.21.15.94:0/2483819048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b54102960 msgr2=0x7f7b541980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53400640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b54102960 0x7f7b541980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53400640 1 -- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b5410bd80 msgr2=0x7f7b5419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53400640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b5410bd80 0x7f7b5419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53400640 1 -- 172.21.15.94:0/2483819048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5419d0a0 con 0x7f7b54103280 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b58e00640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b5410bd80 0x7f7b5419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b53400640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 0x7f7b541985f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e910 tx=0x7f7b3c00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.305+0000 7f7b50c00640 1 -- 172.21.15.94:0/2483819048 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00cd80 con 0x7f7b54103280 2024-06-03T01:17:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.306+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5419d390 con 0x7f7b54103280 2024-06-03T01:17:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.306+0000 7f7b50c00640 1 -- 172.21.15.94:0/2483819048 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3c0035f0 con 0x7f7b54103280 2024-06-03T01:17:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.306+0000 7f7b50c00640 1 -- 172.21.15.94:0/2483819048 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c010670 con 0x7f7b54103280 2024-06-03T01:17:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.306+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b541a4c20 con 0x7f7b54103280 2024-06-03T01:17:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.309+0000 7f7b5dc70740 1 Processor -- start 2024-06-03T01:17:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.309+0000 7f7b5dc70740 1 -- start start 2024-06-03T01:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.309+0000 7f7b50c00640 1 -- 172.21.15.94:0/2483819048 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7b3c0037a0 con 0x7f7b54103280 2024-06-03T01:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca544e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca544ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5459390 0x557ca545b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557ca53ae420 con 0x557ca544fb80 2024-06-03T01:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ca5355c50 con 0x557ca5459390 2024-06-03T01:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557ca53aa3e0 con 0x557ca544e610 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5459390 0x557ca545b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b58e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca544e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca544ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5459390 0x557ca545b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56072/0 (socket says 172.21.15.94:56072) 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 -- 172.21.15.94:0/867332187 learned_addr learned my addr 172.21.15.94:0/867332187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca544ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:44090/0 (socket says 172.21.15.94:44090) 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 -- 172.21.15.94:0/867332187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 msgr2=0x557ca544ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 --2- 172.21.15.94:0/867332187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca544ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 -- 172.21.15.94:0/867332187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 msgr2=0x557ca544e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 --2- 172.21.15.94:0/867332187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca544e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.310+0000 7f7b53400640 1 -- 172.21.15.94:0/867332187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca50c2210 con 0x557ca5459390 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.311+0000 7f7b53400640 1 --2- 172.21.15.94:0/867332187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5459390 0x557ca545b780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0d3ad0 tx=0x7f7b3c0d3f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e82971225332f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.311+0000 7f7b49600640 1 -- 172.21.15.94:0/867332187 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0d8ba0 con 0x557ca5459390 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.311+0000 7f7b49600640 1 -- 172.21.15.94:0/867332187 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3c0e1070 con 0x557ca5459390 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.311+0000 7f7b49600640 1 -- 172.21.15.94:0/867332187 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0db5c0 con 0x557ca5459390 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.311+0000 7f7b50c00640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7b380a2cf0 0x7f7b380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.311+0000 7f7b50c00640 1 -- 172.21.15.94:0/2483819048 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7f7b3c0cfaf0 con 0x7f7b54103280 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b58e00640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7b380a2cf0 0x7f7b380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 -- 172.21.15.94:0/867332187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5459390 msgr2=0x557ca545b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/867332187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5459390 0x557ca545b780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c0d3ad0 tx=0x7f7b3c0d3f90 comp rx=0 tx=0).stop 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 -- 172.21.15.94:0/867332187 shutdown_connections 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/867332187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5459390 0x557ca545b780 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/867332187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca544ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/867332187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca544e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 -- 172.21.15.94:0/867332187 >> 172.21.15.94:0/867332187 conn(0x557ca5412f80 msgr2=0x557ca544ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 -- 172.21.15.94:0/867332187 shutdown_connections 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b5dc70740 1 -- 172.21.15.94:0/867332187 wait complete. 2024-06-03T01:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.312+0000 7f7b58e00640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7b380a2cf0 0x7f7b380a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009cb0 tx=0x7f7b44005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 -- start start 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 Processor -- start 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5456400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca544fb80 0x557ca5456940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca5459390 0x557ca5456e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557ca545c100 con 0x557ca5459390 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ca53690b0 con 0x557ca544fb80 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.313+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557ca545c280 con 0x557ca544e610 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b5dc70740 10 client.0 _ll_register_callbacks cb 0x557ca5354f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca544fb80 0x557ca5456940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5456400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b58e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca5459390 0x557ca5456e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca544fb80 0x557ca5456940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56074/0 (socket says 172.21.15.94:56074) 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 -- 172.21.15.94:0/2307741499 learned_addr learned my addr 172.21.15.94:0/2307741499 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 -- 172.21.15.94:0/2307741499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 msgr2=0x557ca5456400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5456400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 -- 172.21.15.94:0/2307741499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca5459390 msgr2=0x557ca5456e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca5459390 0x557ca5456e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 -- 172.21.15.94:0/2307741499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca518e620 con 0x557ca544fb80 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b53400640 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca544fb80 0x557ca5456940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c09b8f0 tx=0x7f7b3c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c8f2fdec6f41e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.314+0000 7f7b42000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b42000640 1 -- 172.21.15.94:0/2307741499 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c097e90 con 0x557ca544fb80 2024-06-03T01:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b42000640 1 -- 172.21.15.94:0/2307741499 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7f7b3c0cd0a0 con 0x557ca544fb80 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b3c0e1070 con 0x557ca544fb80 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b42000640 1 -- 172.21.15.94:0/2307741499 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7b3c09b550 con 0x557ca544fb80 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 10 client.25859 fetch_fsmap learned FSMap version 37 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557ca54684b0 con 0x557ca544fb80 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b42000640 1 -- 172.21.15.94:0/2307741499 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3c0e1070 con 0x557ca544fb80 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b42000640 1 -- 172.21.15.94:0/2307741499 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f7b3c09be50 con 0x557ca544fb80 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 10 client.25859 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 20 client.25859 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5469800 0x557ca546bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 4 client.25859 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.315+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ca53bbb50 con 0x557ca5469800 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.316+0000 7f7b53e00640 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5469800 0x557ca546bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.316+0000 7f7b53e00640 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5469800 0x557ca546bbf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53fa76b4f1264c0f server_cookie=3a5ee5c86f04c4a1 in_seq=0 out_seq=0 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.316+0000 7f7b42000640 10 client.25859 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b42000640 1 -- 172.21.15.94:0/2307741499 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557ca53bbb50 con 0x557ca5469800 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b42000640 10 client.25859 handle_command_reply: tid=0 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 client.25859 shutdown 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca544fb80 msgr2=0x557ca5456940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca544fb80 0x557ca5456940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c09b8f0 tx=0x7f7b3c000bc0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 shutdown_connections 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca5459390 0x557ca5456e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5469800 0x557ca546bbf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca544fb80 0x557ca5456940 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b42000640 0 client.25859 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/2307741499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5456400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.317+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 >> 172.21.15.94:0/2307741499 conn(0x557ca5412f80 msgr2=0x557ca5450a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.318+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 shutdown_connections 2024-06-03T01:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.318+0000 7f7b5dc70740 1 -- 172.21.15.94:0/2307741499 wait complete. 2024-06-03T01:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.318+0000 7f7b5dc70740 20 client.25859 trim_cache size 0 max 16384 2024-06-03T01:17:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.338+0000 7f7b5dc70740 1 Processor -- start 2024-06-03T01:17:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b5dc70740 1 -- start start 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca544ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5471240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca5471780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541acad0 con 0x557ca544fb80 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541ae610 con 0x557ca5412f80 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541ad1a0 con 0x557ca544e610 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b58e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca5471780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b58e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca5471780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38100/0 (socket says 172.21.15.94:38100) 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b58e00640 1 -- 172.21.15.94:0/3495948738 learned_addr learned my addr 172.21.15.94:0/3495948738 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b53400640 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5471240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.339+0000 7f7b53e00640 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca544ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b58e00640 1 -- 172.21.15.94:0/3495948738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 msgr2=0x557ca5471240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b58e00640 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5471240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b58e00640 1 -- 172.21.15.94:0/3495948738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 msgr2=0x557ca544ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b58e00640 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca544ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b58e00640 1 -- 172.21.15.94:0/3495948738 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca5454950 con 0x557ca544fb80 2024-06-03T01:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b58e00640 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca5471780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b4401f760 tx=0x7f7b4401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3e9a6fd122c5bc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b48c00640 1 -- 172.21.15.94:0/3495948738 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44011580 con 0x557ca544fb80 2024-06-03T01:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b48c00640 1 -- 172.21.15.94:0/3495948738 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b44011c00 con 0x557ca544fb80 2024-06-03T01:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.340+0000 7f7b48c00640 1 -- 172.21.15.94:0/3495948738 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4401dde0 con 0x557ca544fb80 2024-06-03T01:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.341+0000 7f7b5dc70740 1 -- 172.21.15.94:0/3495948738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 msgr2=0x557ca5471780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.341+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca5471780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b4401f760 tx=0x7f7b4401a000 comp rx=0 tx=0).stop 2024-06-03T01:17:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.341+0000 7f7b5dc70740 1 -- 172.21.15.94:0/3495948738 shutdown_connections 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.341+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544fb80 0x557ca5471780 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.341+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544e610 0x557ca5471240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.341+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/3495948738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca544ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.341+0000 7f7b5dc70740 1 -- 172.21.15.94:0/3495948738 >> 172.21.15.94:0/3495948738 conn(0x557ca5451dd0 msgr2=0x557ca544a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.342+0000 7f7b5dc70740 1 -- 172.21.15.94:0/3495948738 shutdown_connections 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.342+0000 7f7b5dc70740 1 -- 172.21.15.94:0/3495948738 wait complete. 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.342+0000 7f7b5dc70740 1 -- start start 2024-06-03T01:17:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.342+0000 7f7b5dc70740 1 Processor -- start 2024-06-03T01:17:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca5461b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544e610 0x557ca545ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b5dc70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544fb80 0x557ca545d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541adf40 con 0x557ca544e610 2024-06-03T01:17:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541ad870 con 0x557ca5412f80 2024-06-03T01:17:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b5dc70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541aaad0 con 0x557ca544fb80 2024-06-03T01:17:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b5dc70740 10 client.0 _ll_register_callbacks cb 0x7f7b44009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca5461b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca5461b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56098/0 (socket says 172.21.15.94:56098) 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 -- 172.21.15.94:0/616995270 learned_addr learned my addr 172.21.15.94:0/616995270 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b58e00640 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544fb80 0x557ca545d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53400640 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544e610 0x557ca545ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 -- 172.21.15.94:0/616995270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544fb80 msgr2=0x557ca545d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544fb80 0x557ca545d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 -- 172.21.15.94:0/616995270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544e610 msgr2=0x557ca545ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544e610 0x557ca545ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.343+0000 7f7b53e00640 1 -- 172.21.15.94:0/616995270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca545da20 con 0x557ca5412f80 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b53e00640 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca5461b00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c02ac70 tx=0x7f7b4c030da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19197f26cd7011ac server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b42000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b42000640 1 -- 172.21.15.94:0/616995270 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c036040 con 0x557ca5412f80 2024-06-03T01:17:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b44011c00 con 0x557ca5412f80 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b42000640 1 -- 172.21.15.94:0/616995270 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7f7b4c0307b0 con 0x557ca5412f80 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b42000640 1 -- 172.21.15.94:0/616995270 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7b4c05b050 con 0x557ca5412f80 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b5dc70740 10 client.25865 fetch_fsmap learned FSMap version 37 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.344+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557ca54d4110 con 0x557ca5412f80 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b42000640 1 -- 172.21.15.94:0/616995270 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b44011c00 con 0x557ca5412f80 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b42000640 1 -- 172.21.15.94:0/616995270 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f7b4c03c040 con 0x557ca5412f80 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b5dc70740 10 client.25865 fetch_fsmap finished waiting for FSMap version 37 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b5dc70740 10 client.25865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b5dc70740 20 client.25865 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5459390 0x557ca54d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b5dc70740 4 client.25865 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:17:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ca53b5ea0 con 0x557ca5459390 2024-06-03T01:17:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b53400640 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5459390 0x557ca54d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.346+0000 7f7b42000640 10 client.25865 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.345+0000 7f7b53400640 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5459390 0x557ca54d67f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d200dc51251d42f server_cookie=fbf9989c8f420756 in_seq=0 out_seq=0 2024-06-03T01:17:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.386+0000 7f7b42000640 1 -- 172.21.15.94:0/616995270 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557ca53b5ea0 con 0x557ca5459390 2024-06-03T01:17:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.386+0000 7f7b42000640 10 client.25865 handle_command_reply: tid=0 2024-06-03T01:17:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.386+0000 7f7b5dc70740 1 client.25865 shutdown 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 msgr2=0x557ca5461b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca5461b00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c02ac70 tx=0x7f7b4c030da0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 shutdown_connections 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ca544fb80 0x557ca545d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557ca544e610 0x557ca545ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557ca5459390 0x557ca54d67f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 --2- 172.21.15.94:0/616995270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca5412f80 0x557ca5461b00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b42000640 0 client.25865 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 >> 172.21.15.94:0/616995270 conn(0x557ca5451dd0 msgr2=0x557ca5450170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 shutdown_connections 2024-06-03T01:17:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 1 -- 172.21.15.94:0/616995270 wait complete. 2024-06-03T01:17:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.387+0000 7f7b5dc70740 20 client.25865 trim_cache size 0 max 16384 2024-06-03T01:17:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:17:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:17:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7b380a2cf0 msgr2=0x7f7b380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7b380a2cf0 0x7f7b380a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009cb0 tx=0x7f7b44005f90 comp rx=0 tx=0).stop 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 msgr2=0x7f7b541985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 0x7f7b541985f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e910 tx=0x7f7b3c00edd0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 shutdown_connections 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7b380a2cf0 0x7f7b380a51b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b5410bd80 0x7f7b5419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b54103280 0x7f7b541985f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 --2- 172.21.15.94:0/2483819048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b54102960 0x7f7b541980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 >> 172.21.15.94:0/2483819048 conn(0x7f7b540fc1c0 msgr2=0x7f7b540fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 shutdown_connections 2024-06-03T01:17:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:27.388+0000 7f7b59800640 1 -- 172.21.15.94:0/2483819048 wait complete. 2024-06-03T01:17:28.099 INFO:tasks.workunit.client.0.smithi094.stdout: 1 162902 2.21 MB/sec execute 215 sec latency 34.284 ms 2024-06-03T01:17:28.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:28 smithi107 ceph-mon[36622]: pgmap v218: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 KiB/s wr, 106 op/s 2024-06-03T01:17:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:28 smithi094 ceph-mon[32119]: pgmap v218: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 KiB/s wr, 106 op/s 2024-06-03T01:17:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:28 smithi179 ceph-mon[37041]: pgmap v218: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 KiB/s wr, 106 op/s 2024-06-03T01:17:29.099 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163604 2.22 MB/sec execute 216 sec latency 29.818 ms 2024-06-03T01:17:30.099 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163906 2.21 MB/sec execute 217 sec latency 515.421 ms 2024-06-03T01:17:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:30 smithi179 ceph-mon[37041]: pgmap v219: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 355 KiB/s wr, 62 op/s 2024-06-03T01:17:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:30 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:31.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:30 smithi107 ceph-mon[36622]: pgmap v219: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 355 KiB/s wr, 62 op/s 2024-06-03T01:17:31.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:30 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:31.100 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163928 2.20 MB/sec execute 218 sec latency 1470.753 ms 2024-06-03T01:17:31.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:30 smithi094 ceph-mon[32119]: pgmap v219: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 355 KiB/s wr, 62 op/s 2024-06-03T01:17:31.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:30 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:32.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:31 smithi107 ceph-mon[36622]: pgmap v220: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 355 KiB/s wr, 62 op/s 2024-06-03T01:17:32.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:31 smithi094 ceph-mon[32119]: pgmap v220: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 355 KiB/s wr, 62 op/s 2024-06-03T01:17:32.099 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163974 2.19 MB/sec execute 219 sec latency 946.074 ms 2024-06-03T01:17:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:31 smithi179 ceph-mon[37041]: pgmap v220: 129 pgs: 129 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 355 KiB/s wr, 62 op/s 2024-06-03T01:17:33.100 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163974 2.18 MB/sec execute 220 sec latency 1946.206 ms 2024-06-03T01:17:34.100 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163974 2.17 MB/sec execute 221 sec latency 2946.341 ms 2024-06-03T01:17:34.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:33 smithi107 ceph-mon[36622]: pgmap v221: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 KiB/s wr, 125 op/s 2024-06-03T01:17:34.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:33 smithi094 ceph-mon[32119]: pgmap v221: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 KiB/s wr, 125 op/s 2024-06-03T01:17:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:33 smithi179 ceph-mon[37041]: pgmap v221: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 873 KiB/s wr, 125 op/s 2024-06-03T01:17:35.100 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163974 2.16 MB/sec execute 222 sec latency 3946.485 ms 2024-06-03T01:17:35.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:35 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:35.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:35 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:35.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:35 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:36.100 INFO:tasks.workunit.client.0.smithi094.stdout: 1 163974 2.15 MB/sec execute 223 sec latency 4946.620 ms 2024-06-03T01:17:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:36 smithi107 ceph-mon[36622]: pgmap v222: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 613 KiB/s wr, 87 op/s 2024-06-03T01:17:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:36 smithi094 ceph-mon[32119]: pgmap v222: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 613 KiB/s wr, 87 op/s 2024-06-03T01:17:36.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:36 smithi179 ceph-mon[37041]: pgmap v222: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 613 KiB/s wr, 87 op/s 2024-06-03T01:17:37.100 INFO:tasks.workunit.client.0.smithi094.stdout: 1 164742 2.16 MB/sec execute 224 sec latency 4949.913 ms 2024-06-03T01:17:38.101 INFO:tasks.workunit.client.0.smithi094.stdout: 1 165790 2.17 MB/sec execute 225 sec latency 117.638 ms 2024-06-03T01:17:38.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:38 smithi107 ceph-mon[36622]: pgmap v223: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 998 KiB/s wr, 119 op/s 2024-06-03T01:17:38.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:38 smithi094 ceph-mon[32119]: pgmap v223: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 998 KiB/s wr, 119 op/s 2024-06-03T01:17:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:38 smithi179 ceph-mon[37041]: pgmap v223: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 998 KiB/s wr, 119 op/s 2024-06-03T01:17:39.100 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167007 2.21 MB/sec execute 226 sec latency 38.528 ms 2024-06-03T01:17:39.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:39 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:17:39.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:39 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:17:39.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:39 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:17:40.101 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167455 2.20 MB/sec execute 227 sec latency 323.942 ms 2024-06-03T01:17:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:40 smithi107 ceph-mon[36622]: pgmap v224: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 95 op/s 2024-06-03T01:17:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:40 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:40.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:40 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:40 smithi094 ceph-mon[32119]: pgmap v224: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 95 op/s 2024-06-03T01:17:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:40 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:40 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:40 smithi179 ceph-mon[37041]: pgmap v224: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 95 op/s 2024-06-03T01:17:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:40 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:40 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 -- 172.21.15.94:0/1888734803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198102960 msgr2=0x7fd198102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 --2- 172.21.15.94:0/1888734803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198102960 0x7fd198102d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009ca0 tx=0x7fd18c02a950 comp rx=0 tx=0).stop 2024-06-03T01:17:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 -- 172.21.15.94:0/1888734803 shutdown_connections 2024-06-03T01:17:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 --2- 172.21.15.94:0/1888734803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19810bd80 0x7fd19810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 --2- 172.21.15.94:0/1888734803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198103280 0x7fd19810b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 --2- 172.21.15.94:0/1888734803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198102960 0x7fd198102d40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 -- 172.21.15.94:0/1888734803 >> 172.21.15.94:0/1888734803 conn(0x7fd1980fc1c0 msgr2=0x7fd1980fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 -- 172.21.15.94:0/1888734803 shutdown_connections 2024-06-03T01:17:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.523+0000 7fd19ea00640 1 -- 172.21.15.94:0/1888734803 wait complete. 2024-06-03T01:17:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.524+0000 7fd19ea00640 1 Processor -- start 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.524+0000 7fd19ea00640 1 -- start start 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198102960 0x7fd1981005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198103280 0x7fd198100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19810bd80 0x7fd198101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd198111bd0 con 0x7fd198103280 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19ea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd198111a50 con 0x7fd19810bd80 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19ea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd198111d50 con 0x7fd198102960 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19d600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198102960 0x7fd1981005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198103280 0x7fd198100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19d600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198102960 0x7fd1981005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51388/0 (socket says 172.21.15.94:51388) 2024-06-03T01:17:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19d600640 1 -- 172.21.15.94:0/107651350 learned_addr learned my addr 172.21.15.94:0/107651350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.525+0000 7fd19e000640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19810bd80 0x7fd198101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19d600640 1 -- 172.21.15.94:0/107651350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19810bd80 msgr2=0x7fd198101030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19d600640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19810bd80 0x7fd198101030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19d600640 1 -- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198103280 msgr2=0x7fd198100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19d600640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198103280 0x7fd198100af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19d600640 1 -- 172.21.15.94:0/107651350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981a6550 con 0x7fd198102960 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19cc00640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198103280 0x7fd198100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19e000640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19810bd80 0x7fd198101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19d600640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198102960 0x7fd1981005b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005e30 tx=0x7fd18c003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd196000640 1 -- 172.21.15.94:0/107651350 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c034040 con 0x7fd198102960 2024-06-03T01:17:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981a67e0 con 0x7fd198102960 2024-06-03T01:17:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd196000640 1 -- 172.21.15.94:0/107651350 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c02fe20 con 0x7fd198102960 2024-06-03T01:17:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd196000640 1 -- 172.21.15.94:0/107651350 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c004800 con 0x7fd198102960 2024-06-03T01:17:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.526+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981a6d20 con 0x7fd198102960 2024-06-03T01:17:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.529+0000 7fd1a2f80740 1 Processor -- start 2024-06-03T01:17:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd1a2f80740 1 -- start start 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd196000640 1 -- 172.21.15.94:0/107651350 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd18c037020 con 0x7fd198102960 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa435cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa436680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa443370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eeaa39a420 con 0x55eeaa440f80 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eeaa341c50 con 0x55eeaa436200 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55eeaa3963e0 con 0x55eeaa437770 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd19e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa435cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd19d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa436680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd19e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa435cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51402/0 (socket says 172.21.15.94:51402) 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd19e000640 1 -- 172.21.15.94:0/2790118933 learned_addr learned my addr 172.21.15.94:0/2790118933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.530+0000 7fd19cc00640 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa443370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd19e000640 1 -- 172.21.15.94:0/2790118933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 msgr2=0x55eeaa436680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd19e000640 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa436680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd19e000640 1 -- 172.21.15.94:0/2790118933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 msgr2=0x55eeaa443370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd19e000640 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa443370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd19e000640 1 -- 172.21.15.94:0/2790118933 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeaa0ae210 con 0x55eeaa437770 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd19cc00640 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa443370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd19e000640 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa435cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd19000bcd0 tx=0x7fd19000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91e4a6ca2d9a2c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:40.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.531+0000 7fd196000640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd1800a2d20 0x7fd1800a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.532+0000 7fd18a600640 1 -- 172.21.15.94:0/2790118933 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190015070 con 0x55eeaa437770 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.532+0000 7fd196000640 1 -- 172.21.15.94:0/107651350 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7fd18c0f0610 con 0x7fd198102960 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.532+0000 7fd19d600640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd1800a2d20 0x7fd1800a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.532+0000 7fd18a600640 1 -- 172.21.15.94:0/2790118933 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd190005500 con 0x55eeaa437770 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.532+0000 7fd18a600640 1 -- 172.21.15.94:0/2790118933 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900116a0 con 0x55eeaa437770 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2790118933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 msgr2=0x55eeaa435cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd19d600640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd1800a2d20 0x7fd1800a51e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55eeaa438ed0 tx=0x7fd18c0f6530 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa435cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd19000bcd0 tx=0x7fd19000f730 comp rx=0 tx=0).stop 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2790118933 shutdown_connections 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa443370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa436680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2790118933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa435cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2790118933 >> 172.21.15.94:0/2790118933 conn(0x55eeaa3fef80 msgr2=0x55eeaa4327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2790118933 shutdown_connections 2024-06-03T01:17:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.533+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2790118933 wait complete. 2024-06-03T01:17:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.534+0000 7fd1a2f80740 1 -- start start 2024-06-03T01:17:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.534+0000 7fd1a2f80740 1 Processor -- start 2024-06-03T01:17:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.534+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa43dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa437770 0x55eeaa43e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa440f80 0x55eeaa43ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55eeaa443d50 con 0x55eeaa437770 2024-06-03T01:17:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eeaa3550b0 con 0x55eeaa436200 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55eeaa443ed0 con 0x55eeaa440f80 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd1a2f80740 10 client.0 _ll_register_callbacks cb 0x55eeaa340f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa437770 0x55eeaa43e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa43dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa43dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:46222/0 (socket says 172.21.15.94:46222) 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 -- 172.21.15.94:0/2142861187 learned_addr learned my addr 172.21.15.94:0/2142861187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa440f80 0x55eeaa43ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 -- 172.21.15.94:0/2142861187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa440f80 msgr2=0x55eeaa43ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa440f80 0x55eeaa43ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 -- 172.21.15.94:0/2142861187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa437770 msgr2=0x55eeaa43e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa437770 0x55eeaa43e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.535+0000 7fd19e000640 1 -- 172.21.15.94:0/2142861187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeaa17a620 con 0x55eeaa436200 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd19e000640 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa43dff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd190004d00 tx=0x7fd190004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfcc69db2b450ff1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd17ea00640 1 -- 172.21.15.94:0/2142861187 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190015040 con 0x55eeaa436200 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd17ea00640 1 -- 172.21.15.94:0/2142861187 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7fd1900051b0 con 0x55eeaa436200 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd190005500 con 0x55eeaa436200 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd17ea00640 1 -- 172.21.15.94:0/2142861187 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd190029460 con 0x55eeaa436200 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd1a2f80740 10 client.25877 fetch_fsmap learned FSMap version 39 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.536+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eeaa450110 con 0x55eeaa436200 2024-06-03T01:17:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd17ea00640 1 -- 172.21.15.94:0/2142861187 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd190005500 con 0x55eeaa436200 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd17ea00640 1 -- 172.21.15.94:0/2142861187 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd190027040 con 0x55eeaa436200 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd1a2f80740 10 client.25877 fetch_fsmap finished waiting for FSMap version 39 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd1a2f80740 10 client.25877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd1a2f80740 20 client.25877 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa451520 0x55eeaa453910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd1a2f80740 4 client.25877 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeaa3a7b50 con 0x55eeaa451520 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.537+0000 7fd19cc00640 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa451520 0x55eeaa453910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.538+0000 7fd19cc00640 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa451520 0x55eeaa453910 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3055b9e5d710a14 server_cookie=a351fee2ab2449b0 in_seq=0 out_seq=0 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.538+0000 7fd17ea00640 10 client.25877 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.538+0000 7fd17ea00640 1 -- 172.21.15.94:0/2142861187 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eeaa3a7b50 con 0x55eeaa451520 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.538+0000 7fd17ea00640 10 client.25877 handle_command_reply: tid=0 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 client.25877 shutdown 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 msgr2=0x55eeaa43dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa43dff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd190004d00 tx=0x7fd190004e00 comp rx=0 tx=0).stop 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 shutdown_connections 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa440f80 0x55eeaa43ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa437770 0x55eeaa43e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa451520 0x55eeaa453910 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2142861187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa43dff0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd17ea00640 0 client.25877 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 >> 172.21.15.94:0/2142861187 conn(0x55eeaa3fef80 msgr2=0x55eeaa438620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 shutdown_connections 2024-06-03T01:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2142861187 wait complete. 2024-06-03T01:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.539+0000 7fd1a2f80740 20 client.25877 trim_cache size 0 max 16384 2024-06-03T01:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 Processor -- start 2024-06-03T01:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 -- start start 2024-06-03T01:17:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa436200 0x55eeaa438470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa437770 0x55eeaa43e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa4590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b0ed0 con 0x55eeaa440f80 2024-06-03T01:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b2ad0 con 0x55eeaa437770 2024-06-03T01:17:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.556+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b15a0 con 0x55eeaa436200 2024-06-03T01:17:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa436200 0x55eeaa438470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa436200 0x55eeaa438470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51424/0 (socket says 172.21.15.94:51424) 2024-06-03T01:17:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 -- 172.21.15.94:0/2785308674 learned_addr learned my addr 172.21.15.94:0/2785308674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa437770 0x55eeaa43e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa4590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 -- 172.21.15.94:0/2785308674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa437770 msgr2=0x55eeaa43e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa437770 0x55eeaa43e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 -- 172.21.15.94:0/2785308674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 msgr2=0x55eeaa4590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa4590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 -- 172.21.15.94:0/2785308674 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeaa436cd0 con 0x55eeaa436200 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.557+0000 7fd19e000640 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa436200 0x55eeaa438470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd19000b620 tx=0x7fd19000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b7f788f4db7137 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.558+0000 7fd189c00640 1 -- 172.21.15.94:0/2785308674 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190015070 con 0x55eeaa436200 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.558+0000 7fd189c00640 1 -- 172.21.15.94:0/2785308674 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd19000fc30 con 0x55eeaa436200 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.558+0000 7fd189c00640 1 -- 172.21.15.94:0/2785308674 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900055a0 con 0x55eeaa436200 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.558+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2785308674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa436200 msgr2=0x55eeaa438470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.558+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa436200 0x55eeaa438470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd19000b620 tx=0x7fd19000f730 comp rx=0 tx=0).stop 2024-06-03T01:17:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2785308674 shutdown_connections 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa4590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa437770 0x55eeaa43e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/2785308674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa436200 0x55eeaa438470 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2785308674 >> 172.21.15.94:0/2785308674 conn(0x55eeaa3fef80 msgr2=0x55eeaa437d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2785308674 shutdown_connections 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 -- 172.21.15.94:0/2785308674 wait complete. 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 -- start start 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.559+0000 7fd1a2f80740 1 Processor -- start 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa4440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa444630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd1a2f80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa444b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b2340 con 0x55eeaa440f80 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b1c70 con 0x55eeaa436200 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd1a2f80740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981aeed0 con 0x55eeaa437770 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd1a2f80740 10 client.0 _ll_register_callbacks cb 0x7fd18c0f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa4440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa444630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa444630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:51430/0 (socket says 172.21.15.94:51430) 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 -- 172.21.15.94:0/3214265136 learned_addr learned my addr 172.21.15.94:0/3214265136 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa444b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 -- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 msgr2=0x55eeaa4440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa4440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 -- 172.21.15.94:0/3214265136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 msgr2=0x55eeaa444b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa444b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19cc00640 1 -- 172.21.15.94:0/3214265136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeaa4ba090 con 0x55eeaa437770 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.560+0000 7fd19e000640 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa4440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd19cc00640 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa444630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd184005d00 tx=0x7fd184012f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb079f9b970acf2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd17ea00640 1 -- 172.21.15.94:0/3214265136 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18401fba0 con 0x55eeaa437770 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd19000fc30 con 0x55eeaa437770 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd17ea00640 1 -- 172.21.15.94:0/3214265136 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7fd18401fd40 con 0x55eeaa437770 2024-06-03T01:17:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd17ea00640 1 -- 172.21.15.94:0/3214265136 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd184005020 con 0x55eeaa437770 2024-06-03T01:17:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd1a2f80740 10 client.25903 fetch_fsmap learned FSMap version 39 2024-06-03T01:17:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eeaa4bb300 con 0x55eeaa437770 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.561+0000 7fd17ea00640 1 -- 172.21.15.94:0/3214265136 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd19000fc30 con 0x55eeaa437770 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd17ea00640 1 -- 172.21.15.94:0/3214265136 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd184029460 con 0x55eeaa437770 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd1a2f80740 10 client.25903 fetch_fsmap finished waiting for FSMap version 39 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd1a2f80740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd1a2f80740 20 client.25903 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa4bbb20 0x55eeaa4bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd1a2f80740 4 client.25903 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eeaa3a1ea0 con 0x55eeaa4bbb20 2024-06-03T01:17:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd19e000640 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa4bbb20 0x55eeaa4bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd19e000640 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa4bbb20 0x55eeaa4bdf40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d77f3d1e674a61 server_cookie=a9f14a5098c96847 in_seq=0 out_seq=0 2024-06-03T01:17:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.562+0000 7fd17ea00640 10 client.25903 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd17ea00640 1 -- 172.21.15.94:0/3214265136 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eeaa3a1ea0 con 0x55eeaa4bbb20 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd17ea00640 10 client.25903 handle_command_reply: tid=0 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 client.25903 shutdown 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 msgr2=0x55eeaa444630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa444630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd184005d00 tx=0x7fd184012f10 comp rx=0 tx=0).stop 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 shutdown_connections 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eeaa440f80 0x55eeaa444b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eeaa437770 0x55eeaa444630 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55eeaa4bbb20 0x55eeaa4bdf40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 --2- 172.21.15.94:0/3214265136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eeaa436200 0x55eeaa4440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd17ea00640 0 client.25903 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.589+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 >> 172.21.15.94:0/3214265136 conn(0x55eeaa3fef80 msgr2=0x55eeaa431970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.590+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 shutdown_connections 2024-06-03T01:17:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.590+0000 7fd1a2f80740 1 -- 172.21.15.94:0/3214265136 wait complete. 2024-06-03T01:17:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.590+0000 7fd1a2f80740 20 client.25903 trim_cache size 0 max 16384 2024-06-03T01:17:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:17:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:17:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:17:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd1800a2d20 msgr2=0x7fd1800a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd1800a2d20 0x7fd1800a51e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55eeaa438ed0 tx=0x7fd18c0f6530 comp rx=0 tx=0).stop 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198102960 msgr2=0x7fd1981005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198102960 0x7fd1981005b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005e30 tx=0x7fd18c003ce0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 shutdown_connections 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd1800a2d20 0x7fd1800a51e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19810bd80 0x7fd198101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd198103280 0x7fd198100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 --2- 172.21.15.94:0/107651350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198102960 0x7fd1981005b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 >> 172.21.15.94:0/107651350 conn(0x7fd1980fc1c0 msgr2=0x7fd1980fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 shutdown_connections 2024-06-03T01:17:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:40.591+0000 7fd19ea00640 1 -- 172.21.15.94:0/107651350 wait complete. 2024-06-03T01:17:41.101 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167487 2.19 MB/sec execute 228 sec latency 1282.749 ms 2024-06-03T01:17:41.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:41 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:17:41.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:41 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:17:41.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:41 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:17:42.101 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167523 2.18 MB/sec execute 229 sec latency 966.832 ms 2024-06-03T01:17:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:42 smithi094 ceph-mon[32119]: pgmap v225: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 95 op/s 2024-06-03T01:17:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:42 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:17:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:42 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:42.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:42 smithi179 ceph-mon[37041]: pgmap v225: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 95 op/s 2024-06-03T01:17:42.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:42 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:17:42.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:42 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:42 smithi107 ceph-mon[36622]: pgmap v225: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 903 KiB/s wr, 95 op/s 2024-06-03T01:17:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:42 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:17:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:42 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:17:43.101 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167523 2.17 MB/sec execute 230 sec latency 1966.929 ms 2024-06-03T01:17:44.101 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167523 2.16 MB/sec execute 231 sec latency 2967.073 ms 2024-06-03T01:17:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:44 smithi094 ceph-mon[32119]: pgmap v226: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-06-03T01:17:44.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:44 smithi179 ceph-mon[37041]: pgmap v226: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-06-03T01:17:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:44 smithi107 ceph-mon[36622]: pgmap v226: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-06-03T01:17:45.101 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167523 2.15 MB/sec execute 232 sec latency 3967.214 ms 2024-06-03T01:17:46.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 167540 2.14 MB/sec execute 233 sec latency 4927.760 ms 2024-06-03T01:17:46.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:46 smithi094 ceph-mon[32119]: pgmap v227: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 940 KiB/s wr, 93 op/s 2024-06-03T01:17:46.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:46 smithi179 ceph-mon[37041]: pgmap v227: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 940 KiB/s wr, 93 op/s 2024-06-03T01:17:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:46 smithi107 ceph-mon[36622]: pgmap v227: 129 pgs: 129 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 940 KiB/s wr, 93 op/s 2024-06-03T01:17:47.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 168424 2.15 MB/sec execute 234 sec latency 69.873 ms 2024-06-03T01:17:48.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 169328 2.16 MB/sec execute 235 sec latency 349.417 ms 2024-06-03T01:17:48.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:48 smithi094 ceph-mon[32119]: pgmap v228: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-06-03T01:17:48.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:48 smithi179 ceph-mon[37041]: pgmap v228: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-06-03T01:17:48.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:48 smithi107 ceph-mon[36622]: pgmap v228: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-06-03T01:17:49.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 170514 2.19 MB/sec execute 236 sec latency 23.183 ms 2024-06-03T01:17:50.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 170939 2.19 MB/sec execute 237 sec latency 154.778 ms 2024-06-03T01:17:50.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:50 smithi094 ceph-mon[32119]: pgmap v229: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 654 KiB/s wr, 79 op/s 2024-06-03T01:17:50.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:50 smithi179 ceph-mon[37041]: pgmap v229: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 654 KiB/s wr, 79 op/s 2024-06-03T01:17:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:50 smithi107 ceph-mon[36622]: pgmap v229: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 654 KiB/s wr, 79 op/s 2024-06-03T01:17:51.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 170969 2.18 MB/sec execute 238 sec latency 1107.509 ms 2024-06-03T01:17:52.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 171882 2.19 MB/sec execute 239 sec latency 20.710 ms 2024-06-03T01:17:52.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:52 smithi094 ceph-mon[32119]: pgmap v230: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 654 KiB/s wr, 79 op/s 2024-06-03T01:17:52.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:52 smithi179 ceph-mon[37041]: pgmap v230: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 654 KiB/s wr, 79 op/s 2024-06-03T01:17:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:52 smithi107 ceph-mon[36622]: pgmap v230: 129 pgs: 129 active+clean; 68 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 654 KiB/s wr, 79 op/s 2024-06-03T01:17:53.102 INFO:tasks.workunit.client.0.smithi094.stdout: 1 172993 2.20 MB/sec execute 240 sec latency 176.702 ms 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 -- 172.21.15.94:0/452824211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 msgr2=0x7f6b2810ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 --2- 172.21.15.94:0/452824211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 0x7f6b2810ba70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009900 tx=0x7f6b1402a480 comp rx=0 tx=0).stop 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 -- 172.21.15.94:0/452824211 shutdown_connections 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 --2- 172.21.15.94:0/452824211 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b2810bfb0 0x7f6b2810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 --2- 172.21.15.94:0/452824211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 0x7f6b2810ba70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 --2- 172.21.15.94:0/452824211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b28102c70 0x7f6b28103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 -- 172.21.15.94:0/452824211 >> 172.21.15.94:0/452824211 conn(0x7f6b28077a20 msgr2=0x7f6b28077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.737+0000 7f6b2f600640 1 -- 172.21.15.94:0/452824211 shutdown_connections 2024-06-03T01:17:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.738+0000 7f6b2f600640 1 -- 172.21.15.94:0/452824211 wait complete. 2024-06-03T01:17:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 Processor -- start 2024-06-03T01:17:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 -- start start 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b28102c70 0x7f6b28198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 0x7f6b28198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2810bfb0 0x7f6b2819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111db0 con 0x7f6b2810bfb0 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111c30 con 0x7f6b28103590 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2f600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111f30 con 0x7f6b28102c70 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2cc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b28102c70 0x7f6b28198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 0x7f6b28198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2cc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b28102c70 0x7f6b28198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:59632/0 (socket says 172.21.15.94:59632) 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.739+0000 7f6b2cc00640 1 -- 172.21.15.94:0/566890639 learned_addr learned my addr 172.21.15.94:0/566890639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2d600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2810bfb0 0x7f6b2819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2cc00640 1 -- 172.21.15.94:0/566890639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 msgr2=0x7f6b28198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 0x7f6b28198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2cc00640 1 -- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2810bfb0 msgr2=0x7f6b2819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2810bfb0 0x7f6b2819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2cc00640 1 -- 172.21.15.94:0/566890639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2819d1d0 con 0x7f6b28102c70 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b28102c70 0x7f6b28198230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ebc0 tx=0x7f6b1800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b25600640 1 -- 172.21.15.94:0/566890639 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18010630 con 0x7f6b28102c70 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b25600640 1 -- 172.21.15.94:0/566890639 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b18010cb0 con 0x7f6b28102c70 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2819d4c0 con 0x7f6b28102c70 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.740+0000 7f6b25600640 1 -- 172.21.15.94:0/566890639 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18018bd0 con 0x7f6b28102c70 2024-06-03T01:17:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.741+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a4d50 con 0x7f6b28102c70 2024-06-03T01:17:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.744+0000 7f6b25600640 1 -- 172.21.15.94:0/566890639 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6b180107d0 con 0x7f6b28102c70 2024-06-03T01:17:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.744+0000 7f6b32428740 1 Processor -- start 2024-06-03T01:17:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.744+0000 7f6b32428740 1 -- start start 2024-06-03T01:17:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b32428740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a94925b0 0x5635a9490b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b32428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a94914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b32428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a949e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b32428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635a93f5420 con 0x5635a9491040 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b32428740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5635a939cc50 con 0x5635a94925b0 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b32428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5635a93f13e0 con 0x5635a949bdc0 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a94925b0 0x5635a9490b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a94925b0 0x5635a9490b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:58884/0 (socket says 172.21.15.94:58884) 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 -- 172.21.15.94:0/1121321481 learned_addr learned my addr 172.21.15.94:0/1121321481 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b27e00640 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a949e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a94914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 -- 172.21.15.94:0/1121321481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 msgr2=0x5635a949e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a949e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 -- 172.21.15.94:0/1121321481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 msgr2=0x5635a94914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a94914c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.745+0000 7f6b2d600640 1 -- 172.21.15.94:0/1121321481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635a9109210 con 0x5635a94925b0 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.746+0000 7f6b25600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f6b0c0a2e20 0x7f6b0c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.746+0000 7f6b2d600640 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a94925b0 0x5635a9490b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009bc0 tx=0x7f6b200117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c531f56993759c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.746+0000 7f6b25600640 1 -- 172.21.15.94:0/566890639 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7f6b180cfec0 con 0x7f6b28102c70 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.746+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f6b0c0a2e20 0x7f6b0c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.746+0000 7f6b1dc00640 1 -- 172.21.15.94:0/1121321481 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2001b470 con 0x5635a94925b0 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.746+0000 7f6b1dc00640 1 -- 172.21.15.94:0/1121321481 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b20016070 con 0x5635a94925b0 2024-06-03T01:17:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.746+0000 7f6b1dc00640 1 -- 172.21.15.94:0/1121321481 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b200129b0 con 0x5635a94925b0 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 -- 172.21.15.94:0/1121321481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a94925b0 msgr2=0x5635a9490b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b2cc00640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f6b0c0a2e20 0x7f6b0c0a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5635a9493d10 tx=0x7f6b1800cf90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a94925b0 0x5635a9490b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009bc0 tx=0x7f6b200117b0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 -- 172.21.15.94:0/1121321481 shutdown_connections 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a949e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a94914c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 --2- 172.21.15.94:0/1121321481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a94925b0 0x5635a9490b00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 -- 172.21.15.94:0/1121321481 >> 172.21.15.94:0/1121321481 conn(0x5635a9459f80 msgr2=0x5635a948d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 -- 172.21.15.94:0/1121321481 shutdown_connections 2024-06-03T01:17:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.747+0000 7f6b32428740 1 -- 172.21.15.94:0/1121321481 wait complete. 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 -- start start 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 Processor -- start 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a9498df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a9499330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a949bdc0 0x5635a9499870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5635a949eb90 con 0x5635a9491040 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5635a93b00b0 con 0x5635a949bdc0 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5635a949ed10 con 0x5635a94925b0 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.748+0000 7f6b32428740 10 client.0 _ll_register_callbacks cb 0x5635a939bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a9499330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a9499330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:59652/0 (socket says 172.21.15.94:59652) 2024-06-03T01:17:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 -- 172.21.15.94:0/862397000 learned_addr learned my addr 172.21.15.94:0/862397000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b2d600640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a9498df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b2d600640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a949bdc0 0x5635a9499870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 -- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a949bdc0 msgr2=0x5635a9499870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a949bdc0 0x5635a9499870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 -- 172.21.15.94:0/862397000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 msgr2=0x5635a9498df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a9498df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 -- 172.21.15.94:0/862397000 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635a91d5620 con 0x5635a94925b0 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b2d600640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a9498df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b27e00640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a9499330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009900 tx=0x7f6b14004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9060a3b62fb7d43f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b12000640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.749+0000 7f6b12000640 1 -- 172.21.15.94:0/862397000 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035040 con 0x5635a94925b0 2024-06-03T01:17:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b12000640 1 -- 172.21.15.94:0/862397000 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7f6b14005450 con 0x5635a94925b0 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b20016070 con 0x5635a94925b0 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b12000640 1 -- 172.21.15.94:0/862397000 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6b1404f050 con 0x5635a94925b0 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 10 client.25915 fetch_fsmap learned FSMap version 40 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5635a94a9d00 con 0x5635a94925b0 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b12000640 1 -- 172.21.15.94:0/862397000 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b20016070 con 0x5635a94925b0 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b12000640 1 -- 172.21.15.94:0/862397000 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f6b14003990 con 0x5635a94925b0 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 10 client.25915 fetch_fsmap finished waiting for FSMap version 40 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 10 client.25915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 20 client.25915 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a94ab210 0x5635a94ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 4 client.25915 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.750+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635a9402b50 con 0x5635a94ab210 2024-06-03T01:17:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.751+0000 7f6b2d600640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a94ab210 0x5635a94ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.751+0000 7f6b2d600640 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a94ab210 0x5635a94ab660 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a223085a4b692b84 server_cookie=fe491bcdea604a84 in_seq=0 out_seq=0 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.751+0000 7f6b12000640 10 client.25915 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b12000640 1 -- 172.21.15.94:0/862397000 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5635a9402b50 con 0x5635a94ab210 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b12000640 10 client.25915 handle_command_reply: tid=0 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 client.25915 shutdown 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 msgr2=0x5635a9499330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a9499330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009900 tx=0x7f6b14004440 comp rx=0 tx=0).stop 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 shutdown_connections 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a94ab210 0x5635a94ab660 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a949bdc0 0x5635a9499870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a9499330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b12000640 0 client.25915 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 --2- 172.21.15.94:0/862397000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a9491040 0x5635a9498df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 >> 172.21.15.94:0/862397000 conn(0x5635a9459f80 msgr2=0x5635a9493460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 shutdown_connections 2024-06-03T01:17:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 1 -- 172.21.15.94:0/862397000 wait complete. 2024-06-03T01:17:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.752+0000 7f6b32428740 20 client.25915 trim_cache size 0 max 16384 2024-06-03T01:17:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.770+0000 7f6b32428740 1 Processor -- start 2024-06-03T01:17:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.770+0000 7f6b32428740 1 -- start start 2024-06-03T01:17:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b32428740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a9497f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b32428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a94a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b32428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a949bdc0 0x5635a94bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b32428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281acbe0 con 0x5635a949bdc0 2024-06-03T01:17:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b32428740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281ae810 con 0x5635a9491040 2024-06-03T01:17:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b32428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281ad2e0 con 0x5635a94925b0 2024-06-03T01:17:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a94a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a9497f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a94a76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:59658/0 (socket says 172.21.15.94:59658) 2024-06-03T01:17:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b27e00640 1 -- 172.21.15.94:0/3818685716 learned_addr learned my addr 172.21.15.94:0/3818685716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.771+0000 7f6b2d600640 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a949bdc0 0x5635a94bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b27e00640 1 -- 172.21.15.94:0/3818685716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 msgr2=0x5635a9497f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b27e00640 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a9497f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b27e00640 1 -- 172.21.15.94:0/3818685716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a949bdc0 msgr2=0x5635a94bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b27e00640 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a949bdc0 0x5635a94bcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b27e00640 1 -- 172.21.15.94:0/3818685716 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635a949ecb0 con 0x5635a94925b0 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b27e00640 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a94a76b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b14000c00 tx=0x7f6b1402ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e1bf082203b007 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b1d200640 1 -- 172.21.15.94:0/3818685716 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14058070 con 0x5635a94925b0 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b1d200640 1 -- 172.21.15.94:0/3818685716 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b14041070 con 0x5635a94925b0 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.772+0000 7f6b1d200640 1 -- 172.21.15.94:0/3818685716 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035030 con 0x5635a94925b0 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 -- 172.21.15.94:0/3818685716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 msgr2=0x5635a94a76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a94a76b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b14000c00 tx=0x7f6b1402ac00 comp rx=0 tx=0).stop 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 -- 172.21.15.94:0/3818685716 shutdown_connections 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a949bdc0 0x5635a94bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a94925b0 0x5635a94a76b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 --2- 172.21.15.94:0/3818685716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a9497f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 -- 172.21.15.94:0/3818685716 >> 172.21.15.94:0/3818685716 conn(0x5635a9459f80 msgr2=0x5635a94a6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 -- 172.21.15.94:0/3818685716 shutdown_connections 2024-06-03T01:17:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 -- 172.21.15.94:0/3818685716 wait complete. 2024-06-03T01:17:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.773+0000 7f6b32428740 1 -- start start 2024-06-03T01:17:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 1 Processor -- start 2024-06-03T01:17:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a94b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a94925b0 0x5635a94b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a94b8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281ae080 con 0x5635a94925b0 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281ad9b0 con 0x5635a9491040 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281aabe0 con 0x5635a949bdc0 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b32428740 10 client.0 _ll_register_callbacks cb 0x7f6b14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a94b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b2d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a94b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:58912/0 (socket says 172.21.15.94:58912) 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b2d600640 1 -- 172.21.15.94:0/421966601 learned_addr learned my addr 172.21.15.94:0/421966601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.774+0000 7f6b27e00640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a94925b0 0x5635a94b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b2d600640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a94b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b2d600640 1 -- 172.21.15.94:0/421966601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 msgr2=0x5635a94b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b2d600640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a94b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b2d600640 1 -- 172.21.15.94:0/421966601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a94925b0 msgr2=0x5635a94b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b2d600640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a94925b0 0x5635a94b8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b2d600640 1 -- 172.21.15.94:0/421966601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635a951e060 con 0x5635a9491040 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b27e00640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a94925b0 0x5635a94b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b2d600640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a94b7ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009b90 tx=0x7f6b200117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29573de217e33708 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.775+0000 7f6b12000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b12000640 1 -- 172.21.15.94:0/421966601 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b200055e0 con 0x5635a9491040 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b14041070 con 0x5635a9491040 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b12000640 1 -- 172.21.15.94:0/421966601 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9475+0+0 (secure 0 0 0) 0x7f6b200126c0 con 0x5635a9491040 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b12000640 1 -- 172.21.15.94:0/421966601 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6b200128c0 con 0x5635a9491040 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 10 client.25901 fetch_fsmap learned FSMap version 40 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5635a951f370 con 0x5635a9491040 2024-06-03T01:17:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b12000640 1 -- 172.21.15.94:0/421966601 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b14041070 con 0x5635a9491040 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b12000640 1 -- 172.21.15.94:0/421966601 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f6b20029070 con 0x5635a9491040 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 10 client.25901 fetch_fsmap finished waiting for FSMap version 40 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 10 client.25901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 20 client.25901 populate_metadata read hostname 'smithi094' 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a951f880 0x5635a9521ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 4 client.25901 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.776+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635a93fcea0 con 0x5635a951f880 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.777+0000 7f6b27e00640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a951f880 0x5635a9521ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.777+0000 7f6b12000640 10 client.25901 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.777+0000 7f6b27e00640 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a951f880 0x5635a9521ca0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824dccaafa3afba2 server_cookie=5efee90be6f1e982 in_seq=0 out_seq=0 2024-06-03T01:17:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b12000640 1 -- 172.21.15.94:0/421966601 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5635a93fcea0 con 0x5635a951f880 2024-06-03T01:17:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b12000640 10 client.25901 handle_command_reply: tid=0 2024-06-03T01:17:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 client.25901 shutdown 2024-06-03T01:17:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 msgr2=0x5635a94b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a94b7ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009b90 tx=0x7f6b200117b0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 shutdown_connections 2024-06-03T01:17:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635a949bdc0 0x5635a94b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5635a94925b0 0x5635a94b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5635a951f880 0x5635a9521ca0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 --2- 172.21.15.94:0/421966601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635a9491040 0x5635a94b7ef0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 >> 172.21.15.94:0/421966601 conn(0x5635a9459f80 msgr2=0x5635a9498db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.806+0000 7f6b12000640 0 client.25901 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:17:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.807+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 shutdown_connections 2024-06-03T01:17:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.807+0000 7f6b32428740 1 -- 172.21.15.94:0/421966601 wait complete. 2024-06-03T01:17:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.807+0000 7f6b32428740 20 client.25901 trim_cache size 0 max 16384 2024-06-03T01:17:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:17:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:17:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f6b0c0a2e20 msgr2=0x7f6b0c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f6b0c0a2e20 0x7f6b0c0a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5635a9493d10 tx=0x7f6b1800cf90 comp rx=0 tx=0).stop 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b28102c70 msgr2=0x7f6b28198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b28102c70 0x7f6b28198230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ebc0 tx=0x7f6b1800c6a0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 shutdown_connections 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f6b0c0a2e20 0x7f6b0c0a52e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b2810bfb0 0x7f6b2819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b28103590 0x7f6b28198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 --2- 172.21.15.94:0/566890639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b28102c70 0x7f6b28198230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:17:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 >> 172.21.15.94:0/566890639 conn(0x7f6b28077a20 msgr2=0x7f6b280fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:17:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 shutdown_connections 2024-06-03T01:17:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:17:53.808+0000 7f6b2f600640 1 -- 172.21.15.94:0/566890639 wait complete. 2024-06-03T01:17:54.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174099 2.23 MB/sec execute 241 sec latency 17.375 ms 2024-06-03T01:17:54.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:54 smithi094 ceph-mon[32119]: pgmap v231: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-03T01:17:54.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:54 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:54.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:54 smithi179 ceph-mon[37041]: pgmap v231: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-03T01:17:54.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:54 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:54 smithi107 ceph-mon[36622]: pgmap v231: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-03T01:17:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:54 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:55.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174489 2.22 MB/sec execute 242 sec latency 170.432 ms 2024-06-03T01:17:56.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174506 2.21 MB/sec execute 243 sec latency 1128.784 ms 2024-06-03T01:17:56.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:56 smithi094 ceph-mon[32119]: pgmap v232: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 654 KiB/s wr, 80 op/s 2024-06-03T01:17:56.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:56 smithi179 ceph-mon[37041]: pgmap v232: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 654 KiB/s wr, 80 op/s 2024-06-03T01:17:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:56 smithi107 ceph-mon[36622]: pgmap v232: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 654 KiB/s wr, 80 op/s 2024-06-03T01:17:57.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174545 2.20 MB/sec execute 244 sec latency 944.172 ms 2024-06-03T01:17:58.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174545 2.19 MB/sec execute 245 sec latency 1944.277 ms 2024-06-03T01:17:58.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:58 smithi094 ceph-mon[32119]: pgmap v233: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 918 KiB/s wr, 130 op/s 2024-06-03T01:17:58.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:58 smithi179 ceph-mon[37041]: pgmap v233: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 918 KiB/s wr, 130 op/s 2024-06-03T01:17:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:58 smithi107 ceph-mon[36622]: pgmap v233: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 918 KiB/s wr, 130 op/s 2024-06-03T01:17:59.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174545 2.18 MB/sec execute 246 sec latency 2944.447 ms 2024-06-03T01:17:59.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:17:59 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:17:59 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:17:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:17:59 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:18:00.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174545 2.18 MB/sec execute 247 sec latency 3944.593 ms 2024-06-03T01:18:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:00 smithi107 ceph-mon[36622]: pgmap v234: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 110 op/s 2024-06-03T01:18:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:00 smithi094 ceph-mon[32119]: pgmap v234: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 110 op/s 2024-06-03T01:18:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:00 smithi179 ceph-mon[37041]: pgmap v234: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 110 op/s 2024-06-03T01:18:01.103 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174697 2.17 MB/sec execute 248 sec latency 4749.348 ms 2024-06-03T01:18:02.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 174854 2.16 MB/sec execute 249 sec latency 96.694 ms 2024-06-03T01:18:02.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:02 smithi107 ceph-mon[36622]: pgmap v235: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 110 op/s 2024-06-03T01:18:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:02 smithi094 ceph-mon[32119]: pgmap v235: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 110 op/s 2024-06-03T01:18:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:02 smithi179 ceph-mon[37041]: pgmap v235: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 110 op/s 2024-06-03T01:18:03.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 175471 2.17 MB/sec execute 250 sec latency 92.831 ms 2024-06-03T01:18:04.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 176213 2.17 MB/sec execute 251 sec latency 481.317 ms 2024-06-03T01:18:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:04 smithi107 ceph-mon[36622]: pgmap v236: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-06-03T01:18:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:04 smithi094 ceph-mon[32119]: pgmap v236: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-06-03T01:18:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:04 smithi179 ceph-mon[37041]: pgmap v236: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-06-03T01:18:05.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 177196 2.19 MB/sec execute 252 sec latency 49.954 ms 2024-06-03T01:18:05.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:05 smithi107 ceph-mon[36622]: osdmap e104: 12 total, 12 up, 12 in 2024-06-03T01:18:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:05 smithi094 ceph-mon[32119]: osdmap e104: 12 total, 12 up, 12 in 2024-06-03T01:18:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:05 smithi179 ceph-mon[37041]: osdmap e104: 12 total, 12 up, 12 in 2024-06-03T01:18:06.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 177835 2.20 MB/sec execute 253 sec latency 17.667 ms 2024-06-03T01:18:06.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:06 smithi107 ceph-mon[36622]: pgmap v238: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 794 KiB/s wr, 118 op/s 2024-06-03T01:18:06.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:06 smithi094 ceph-mon[32119]: pgmap v238: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 794 KiB/s wr, 118 op/s 2024-06-03T01:18:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:06 smithi179 ceph-mon[37041]: pgmap v238: 129 pgs: 129 active+clean; 71 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 794 KiB/s wr, 118 op/s 2024-06-03T01:18:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2806867565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd01017b0 msgr2=0x7fdbd0101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2806867565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd01017b0 0x7fdbd0101c30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009900 tx=0x7fdbbc02a410 comp rx=0 tx=0).stop 2024-06-03T01:18:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2806867565 shutdown_connections 2024-06-03T01:18:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2806867565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd0102170 0x7fdbd010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2806867565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd01017b0 0x7fdbd0101c30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2806867565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbd01077c0 0x7fdbd0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2806867565 >> 172.21.15.94:0/2806867565 conn(0x7fdbd00fd2e0 msgr2=0x7fdbd00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2806867565 shutdown_connections 2024-06-03T01:18:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.960+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2806867565 wait complete. 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.961+0000 7fdbd6c00640 1 Processor -- start 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.961+0000 7fdbd6c00640 1 -- start start 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd6c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd01017b0 0x7fdbd01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd6c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd0102170 0x7fdbd0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd6c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbd01077c0 0x7fdbd019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd6c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0111c20 con 0x7fdbd01077c0 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd6c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0111aa0 con 0x7fdbd0102170 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd6c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0111da0 con 0x7fdbd01017b0 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd01017b0 0x7fdbd01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd0102170 0x7fdbd0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd0102170 0x7fdbd0198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:55886/0 (socket says 172.21.15.94:55886) 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd01017b0 0x7fdbd01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53238/0 (socket says 172.21.15.94:53238) 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd4e00640 1 -- 172.21.15.94:0/2415348782 learned_addr learned my addr 172.21.15.94:0/2415348782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.962+0000 7fdbd6200640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbd01077c0 0x7fdbd019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd4e00640 1 -- 172.21.15.94:0/2415348782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd01017b0 msgr2=0x7fdbd01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd01017b0 0x7fdbd01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd4e00640 1 -- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbd01077c0 msgr2=0x7fdbd019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbd01077c0 0x7fdbd019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd4e00640 1 -- 172.21.15.94:0/2415348782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd019d0b0 con 0x7fdbd0102170 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd5800640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd01017b0 0x7fdbd01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd0102170 0x7fdbd0198600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc02aa00 tx=0x7fdbbc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbce000640 1 -- 172.21.15.94:0/2415348782 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc03f070 con 0x7fdbd0102170 2024-06-03T01:18:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbce000640 1 -- 172.21.15.94:0/2415348782 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbbc031c30 con 0x7fdbd0102170 2024-06-03T01:18:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd019d340 con 0x7fdbd0102170 2024-06-03T01:18:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbce000640 1 -- 172.21.15.94:0/2415348782 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc035030 con 0x7fdbd0102170 2024-06-03T01:18:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.963+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd0073bd0 con 0x7fdbd0102170 2024-06-03T01:18:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 Processor -- start 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 -- start start 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbce000640 1 -- 172.21.15.94:0/2415348782 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdbbc042070 con 0x7fdbd0102170 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7a200 0x55b3ead7a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead84f80 0x55b3ead87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3eacde420 con 0x55b3ead84f80 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b3eac85c50 con 0x55b3ead7b770 2024-06-03T01:18:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b3eacda3e0 con 0x55b3ead7a200 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbd6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbd5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7a200 0x55b3ead7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbd5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7a200 0x55b3ead7a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53242/0 (socket says 172.21.15.94:53242) 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbd6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead79cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:55890/0 (socket says 172.21.15.94:55890) 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.967+0000 7fdbd5800640 1 -- 172.21.15.94:0/3004870465 learned_addr learned my addr 172.21.15.94:0/3004870465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.968+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead84f80 0x55b3ead87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.968+0000 7fdbd5800640 1 -- 172.21.15.94:0/3004870465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 msgr2=0x55b3ead79cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.968+0000 7fdbd5800640 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead79cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.968+0000 7fdbd5800640 1 -- 172.21.15.94:0/3004870465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead84f80 msgr2=0x55b3ead87370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.968+0000 7fdbd5800640 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead84f80 0x55b3ead87370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.968+0000 7fdbd5800640 1 -- 172.21.15.94:0/3004870465 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ea9f2210 con 0x55b3ead7a200 2024-06-03T01:18:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.968+0000 7fdbd5800640 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7a200 0x55b3ead7a680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009720 tx=0x7fdbc4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc1448b05f12b658 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbc2a00640 1 -- 172.21.15.94:0/3004870465 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc401e020 con 0x55b3ead7a200 2024-06-03T01:18:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbc2a00640 1 -- 172.21.15.94:0/3004870465 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc400bd80 con 0x55b3ead7a200 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbc2a00640 1 -- 172.21.15.94:0/3004870465 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4019360 con 0x55b3ead7a200 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbce000640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdbb80a2cf0 0x7fdbb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbce000640 1 -- 172.21.15.94:0/2415348782 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9563+0+0 (secure 0 0 0) 0x7fdbbc0efea0 con 0x7fdbd0102170 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbd6200640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdbb80a2cf0 0x7fdbb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbdaff0740 1 -- 172.21.15.94:0/3004870465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7a200 msgr2=0x55b3ead7a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.969+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7a200 0x55b3ead7a680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009720 tx=0x7fdbc4005f90 comp rx=0 tx=0).stop 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 -- 172.21.15.94:0/3004870465 shutdown_connections 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead84f80 0x55b3ead87370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7a200 0x55b3ead7a680 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/3004870465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 -- 172.21.15.94:0/3004870465 >> 172.21.15.94:0/3004870465 conn(0x55b3ead42f80 msgr2=0x55b3ead767f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 -- 172.21.15.94:0/3004870465 shutdown_connections 2024-06-03T01:18:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbd6200640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdbb80a2cf0 0x7fdbb80a51b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005fd0 tx=0x7fdbc8009500 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 -- 172.21.15.94:0/3004870465 wait complete. 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.970+0000 7fdbdaff0740 1 -- start start 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.971+0000 7fdbdaff0740 1 Processor -- start 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.971+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead81ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.971+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7b770 0x55b3ead82430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.971+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead84f80 0x55b3ead82970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.971+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3ead87d50 con 0x55b3ead7a200 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.971+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b3eac990b0 con 0x55b3ead84f80 2024-06-03T01:18:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.971+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b3ead87ed0 con 0x55b3ead7b770 2024-06-03T01:18:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7b770 0x55b3ead82430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead81ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7b770 0x55b3ead82430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53254/0 (socket says 172.21.15.94:53254) 2024-06-03T01:18:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 -- 172.21.15.94:0/605997998 learned_addr learned my addr 172.21.15.94:0/605997998 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbdaff0740 10 client.0 _ll_register_callbacks cb 0x55b3eac84f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd6200640 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead84f80 0x55b3ead82970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 -- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead84f80 msgr2=0x55b3ead82970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead84f80 0x55b3ead82970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 -- 172.21.15.94:0/605997998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 msgr2=0x55b3ead81ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead81ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.972+0000 7fdbd4e00640 1 -- 172.21.15.94:0/605997998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3eaabe620 con 0x55b3ead7b770 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7b770 0x55b3ead82430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0ed1e0 tx=0x7fdbbc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ac6ee616990d86 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbb7400640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbb7400640 1 -- 172.21.15.94:0/605997998 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc049070 con 0x55b3ead7b770 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc400bd80 con 0x55b3ead7b770 2024-06-03T01:18:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbb7400640 1 -- 172.21.15.94:0/605997998 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9563+0+0 (secure 0 0 0) 0x7fdbbc0bc260 con 0x55b3ead7b770 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbb7400640 1 -- 172.21.15.94:0/605997998 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbbc0bbdd0 con 0x55b3ead7b770 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbdaff0740 10 client.25933 fetch_fsmap learned FSMap version 41 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b3ead87e00 con 0x55b3ead7b770 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.973+0000 7fdbb7400640 1 -- 172.21.15.94:0/605997998 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbc400bd80 con 0x55b3ead7b770 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbb7400640 1 -- 172.21.15.94:0/605997998 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fdbbc0c4730 con 0x55b3ead7b770 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbdaff0740 10 client.25933 fetch_fsmap finished waiting for FSMap version 41 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbdaff0740 10 client.25933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbdaff0740 20 client.25933 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3ead95530 0x55b3ead979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbdaff0740 4 client.25933 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3eacebb50 con 0x55b3ead95530 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbd5800640 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3ead95530 0x55b3ead979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbd5800640 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3ead95530 0x55b3ead979d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b3ba718c5fc258 server_cookie=eb72745aa0a3b54e in_seq=0 out_seq=0 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.974+0000 7fdbb7400640 10 client.25933 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.975+0000 7fdbb7400640 1 -- 172.21.15.94:0/605997998 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b3eacebb50 con 0x55b3ead95530 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.975+0000 7fdbb7400640 10 client.25933 handle_command_reply: tid=0 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.975+0000 7fdbdaff0740 1 client.25933 shutdown 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7b770 msgr2=0x55b3ead82430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7b770 0x55b3ead82430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0ed1e0 tx=0x7fdbbc009260 comp rx=0 tx=0).stop 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 shutdown_connections 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3ead95530 0x55b3ead979d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead84f80 0x55b3ead82970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead7b770 0x55b3ead82430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/605997998 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead81ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 >> 172.21.15.94:0/605997998 conn(0x55b3ead42f80 msgr2=0x55b3ead7c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbb7400640 0 client.25933 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 shutdown_connections 2024-06-03T01:18:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 1 -- 172.21.15.94:0/605997998 wait complete. 2024-06-03T01:18:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.976+0000 7fdbdaff0740 20 client.25933 trim_cache size 0 max 16384 2024-06-03T01:18:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 Processor -- start 2024-06-03T01:18:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 -- start start 2024-06-03T01:18:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead80e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead9d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01acb80 con 0x55b3ead7a200 2024-06-03T01:18:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01ae7e0 con 0x55b3ead7b770 2024-06-03T01:18:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.997+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01ad2b0 con 0x55b3ead84f80 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd6200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead9d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead757f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:55912/0 (socket says 172.21.15.94:55912) 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 -- 172.21.15.94:0/4174907659 learned_addr learned my addr 172.21.15.94:0/4174907659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd5800640 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead80e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 -- 172.21.15.94:0/4174907659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 msgr2=0x55b3ead9d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead9d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 -- 172.21.15.94:0/4174907659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 msgr2=0x55b3ead80e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead80e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 -- 172.21.15.94:0/4174907659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ead773c0 con 0x55b3ead7b770 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd6200640 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead9d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead757f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0bbd00 tx=0x7fdbbc0dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e694a1f6b5e4eec server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.998+0000 7fdbc2000640 1 -- 172.21.15.94:0/4174907659 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc049070 con 0x55b3ead7b770 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbc2000640 1 -- 172.21.15.94:0/4174907659 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbbc03b460 con 0x55b3ead7b770 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbc2000640 1 -- 172.21.15.94:0/4174907659 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0bb960 con 0x55b3ead7b770 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 -- 172.21.15.94:0/4174907659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 msgr2=0x55b3ead757f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead757f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0bbd00 tx=0x7fdbbc0dc040 comp rx=0 tx=0).stop 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 -- 172.21.15.94:0/4174907659 shutdown_connections 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead9d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead757f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/4174907659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead80e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 -- 172.21.15.94:0/4174907659 >> 172.21.15.94:0/4174907659 conn(0x55b3ead42f80 msgr2=0x55b3ead750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 -- 172.21.15.94:0/4174907659 shutdown_connections 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:06.999+0000 7fdbdaff0740 1 -- 172.21.15.94:0/4174907659 wait complete. 2024-06-03T01:18:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.000+0000 7fdbdaff0740 1 -- start start 2024-06-03T01:18:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.000+0000 7fdbdaff0740 1 Processor -- start 2024-06-03T01:18:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead8d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead7ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbdaff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01ae050 con 0x55b3ead7a200 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01ad980 con 0x55b3ead7b770 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbdaff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01aab80 con 0x55b3ead84f80 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbdaff0740 10 client.0 _ll_register_callbacks cb 0x7fdbbc0f3640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead8d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead7ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead8d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47370/0 (socket says 172.21.15.94:47370) 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 -- 172.21.15.94:0/1173365634 learned_addr learned my addr 172.21.15.94:0/1173365634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd6200640 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead7ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 -- 172.21.15.94:0/1173365634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 msgr2=0x55b3ead7ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead7ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 -- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 msgr2=0x55b3ead7ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead7ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd5800640 1 -- 172.21.15.94:0/1173365634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ead7f540 con 0x55b3ead7a200 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.001+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead7ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbd5800640 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead8d6a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdbc401aaf0 tx=0x7fdbc401aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abcb57f4b6e1907 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbb7400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbb7400640 1 -- 172.21.15.94:0/1173365634 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc401e040 con 0x55b3ead7a200 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3eace5ea0 con 0x55b3ead7a200 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbb7400640 1 -- 172.21.15.94:0/1173365634 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9563+0+0 (secure 0 0 0) 0x7fdbc40050c0 con 0x55b3ead7a200 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbb7400640 1 -- 172.21.15.94:0/1173365634 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbc4038050 con 0x55b3ead7a200 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbdaff0740 10 client.15804 fetch_fsmap learned FSMap version 41 2024-06-03T01:18:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.002+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b3eae00040 con 0x55b3ead7a200 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbb7400640 1 -- 172.21.15.94:0/1173365634 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b3eace5ea0 con 0x55b3ead7a200 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbb7400640 1 -- 172.21.15.94:0/1173365634 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fdbc40035f0 con 0x55b3ead7a200 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbdaff0740 10 client.15804 fetch_fsmap finished waiting for FSMap version 41 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbdaff0740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbdaff0740 20 client.15804 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3eae00530 0x55b3eae02950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbdaff0740 4 client.15804 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3eac637a0 con 0x55b3eae00530 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.003+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3eae00530 0x55b3eae02950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.004+0000 7fdbd4e00640 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3eae00530 0x55b3eae02950 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3cef63101d19681 server_cookie=e51ff9ea1824b8cd in_seq=0 out_seq=0 2024-06-03T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.004+0000 7fdbb7400640 10 client.15804 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.028+0000 7fdbb7400640 1 -- 172.21.15.94:0/1173365634 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b3eac637a0 con 0x55b3eae00530 2024-06-03T01:18:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.028+0000 7fdbb7400640 10 client.15804 handle_command_reply: tid=0 2024-06-03T01:18:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.028+0000 7fdbdaff0740 1 client.15804 shutdown 2024-06-03T01:18:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 msgr2=0x55b3ead8d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead8d6a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdbc401aaf0 tx=0x7fdbc401aa80 comp rx=0 tx=0).stop 2024-06-03T01:18:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 shutdown_connections 2024-06-03T01:18:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b3ead84f80 0x55b3ead7ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b3eae00530 0x55b3eae02950 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b3ead7b770 0x55b3ead7ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 --2- 172.21.15.94:0/1173365634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3ead7a200 0x55b3ead8d6a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbb7400640 0 client.15804 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 >> 172.21.15.94:0/1173365634 conn(0x55b3ead42f80 msgr2=0x55b3ead7bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 shutdown_connections 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 1 -- 172.21.15.94:0/1173365634 wait complete. 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.029+0000 7fdbdaff0740 20 client.15804 trim_cache size 0 max 16384 2024-06-03T01:18:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:18:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:18:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:18:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdbb80a2cf0 msgr2=0x7fdbb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdbb80a2cf0 0x7fdbb80a51b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005fd0 tx=0x7fdbc8009500 comp rx=0 tx=0).stop 2024-06-03T01:18:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd0102170 msgr2=0x7fdbd0198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd0102170 0x7fdbd0198600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc02aa00 tx=0x7fdbbc003c10 comp rx=0 tx=0).stop 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 shutdown_connections 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdbb80a2cf0 0x7fdbb80a51b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbd01077c0 0x7fdbd019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd0102170 0x7fdbd0198600 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 --2- 172.21.15.94:0/2415348782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbd01017b0 0x7fdbd01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 >> 172.21.15.94:0/2415348782 conn(0x7fdbd00fd2e0 msgr2=0x7fdbd00fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.030+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 shutdown_connections 2024-06-03T01:18:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:07.031+0000 7fdbd6c00640 1 -- 172.21.15.94:0/2415348782 wait complete. 2024-06-03T01:18:07.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 178766 2.21 MB/sec execute 254 sec latency 26.302 ms 2024-06-03T01:18:08.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 179058 2.20 MB/sec execute 255 sec latency 382.938 ms 2024-06-03T01:18:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:08 smithi107 ceph-mon[36622]: pgmap v239: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 941 KiB/s wr, 131 op/s 2024-06-03T01:18:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:08 smithi094 ceph-mon[32119]: pgmap v239: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 941 KiB/s wr, 131 op/s 2024-06-03T01:18:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:08 smithi179 ceph-mon[37041]: pgmap v239: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 941 KiB/s wr, 131 op/s 2024-06-03T01:18:09.105 INFO:tasks.workunit.client.0.smithi094.stdout: 1 180124 2.22 MB/sec execute 256 sec latency 400.568 ms 2024-06-03T01:18:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:09 smithi107 ceph-mon[36622]: osdmap e105: 12 total, 12 up, 12 in 2024-06-03T01:18:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:09 smithi107 ceph-mon[36622]: pgmap v241: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-03T01:18:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:09 smithi094 ceph-mon[32119]: osdmap e105: 12 total, 12 up, 12 in 2024-06-03T01:18:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:09 smithi094 ceph-mon[32119]: pgmap v241: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-03T01:18:10.104 INFO:tasks.workunit.client.0.smithi094.stdout: 1 181221 2.24 MB/sec execute 257 sec latency 22.206 ms 2024-06-03T01:18:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:09 smithi179 ceph-mon[37041]: osdmap e105: 12 total, 12 up, 12 in 2024-06-03T01:18:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:09 smithi179 ceph-mon[37041]: pgmap v241: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-06-03T01:18:11.105 INFO:tasks.workunit.client.0.smithi094.stdout: 1 181732 2.23 MB/sec execute 258 sec latency 98.869 ms 2024-06-03T01:18:12.105 INFO:tasks.workunit.client.0.smithi094.stdout: 1 182604 2.24 MB/sec execute 259 sec latency 158.774 ms 2024-06-03T01:18:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:12 smithi107 ceph-mon[36622]: pgmap v242: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 580 KiB/s wr, 91 op/s 2024-06-03T01:18:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:12 smithi094 ceph-mon[32119]: pgmap v242: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 580 KiB/s wr, 91 op/s 2024-06-03T01:18:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:12 smithi179 ceph-mon[37041]: pgmap v242: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 580 KiB/s wr, 91 op/s 2024-06-03T01:18:13.105 INFO:tasks.workunit.client.0.smithi094.stdout: 1 182604 2.23 MB/sec execute 260 sec latency 1158.874 ms 2024-06-03T01:18:14.105 INFO:tasks.workunit.client.0.smithi094.stdout: 1 182604 2.22 MB/sec execute 261 sec latency 2158.974 ms 2024-06-03T01:18:14.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:14 smithi107 ceph-mon[36622]: pgmap v243: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-06-03T01:18:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:14 smithi094 ceph-mon[32119]: pgmap v243: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-06-03T01:18:14.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:14 smithi179 ceph-mon[37041]: pgmap v243: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-06-03T01:18:15.105 INFO:tasks.workunit.client.0.smithi094.stdout: 1 182604 2.22 MB/sec execute 262 sec latency 3159.133 ms 2024-06-03T01:18:16.105 INFO:tasks.workunit.client.0.smithi094.stdout: 1 183813 2.23 MB/sec execute 263 sec latency 3371.346 ms 2024-06-03T01:18:16.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:16 smithi107 ceph-mon[36622]: pgmap v244: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-03T01:18:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:16 smithi094 ceph-mon[32119]: pgmap v244: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-03T01:18:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:16 smithi179 ceph-mon[37041]: pgmap v244: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-06-03T01:18:17.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 184826 2.25 MB/sec execute 264 sec latency 17.392 ms 2024-06-03T01:18:18.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 185592 2.25 MB/sec execute 265 sec latency 19.618 ms 2024-06-03T01:18:18.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:18 smithi107 ceph-mon[36622]: pgmap v245: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-06-03T01:18:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:18 smithi094 ceph-mon[32119]: pgmap v245: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-06-03T01:18:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:18 smithi179 ceph-mon[37041]: pgmap v245: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-06-03T01:18:19.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 186148 2.25 MB/sec execute 266 sec latency 379.083 ms 2024-06-03T01:18:20.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 186148 2.25 MB/sec execute 267 sec latency 1379.186 ms 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 -- 172.21.15.94:0/318808827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680106890 msgr2=0x7f4680106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 --2- 172.21.15.94:0/318808827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680106890 0x7f4680106c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4674009860 tx=0x7f467402a340 comp rx=0 tx=0).stop 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 -- 172.21.15.94:0/318808827 shutdown_connections 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 --2- 172.21.15.94:0/318808827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4680101290 0x7f468010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 --2- 172.21.15.94:0/318808827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46801008d0 0x7f4680100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 --2- 172.21.15.94:0/318808827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680106890 0x7f4680106c70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 -- 172.21.15.94:0/318808827 >> 172.21.15.94:0/318808827 conn(0x7f4680077a20 msgr2=0x7f4680077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 -- 172.21.15.94:0/318808827 shutdown_connections 2024-06-03T01:18:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.174+0000 7f4687c00640 1 -- 172.21.15.94:0/318808827 wait complete. 2024-06-03T01:18:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.175+0000 7f4687c00640 1 Processor -- start 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.175+0000 7f4687c00640 1 -- start start 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4687c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46801008d0 0x7f468019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4687c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680101290 0x7f468019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4687c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4680106890 0x7f468019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4687c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4680110ca0 con 0x7f4680106890 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4687c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4680110b20 con 0x7f4680101290 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4687c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4680110e20 con 0x7f46801008d0 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4685c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4680106890 0x7f468019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4685c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4680106890 0x7f468019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50734/0 (socket says 172.21.15.94:50734) 2024-06-03T01:18:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4685c00640 1 -- 172.21.15.94:0/3529405932 learned_addr learned my addr 172.21.15.94:0/3529405932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f4685200640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46801008d0 0x7f468019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.176+0000 7f467fe00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680101290 0x7f468019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4685200640 1 -- 172.21.15.94:0/3529405932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680101290 msgr2=0x7f468019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4685200640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680101290 0x7f468019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4685200640 1 -- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4680106890 msgr2=0x7f468019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4685200640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4680106890 0x7f468019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4685200640 1 -- 172.21.15.94:0/3529405932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468019f390 con 0x7f46801008d0 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4685c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4680106890 0x7f468019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4685200640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46801008d0 0x7f468019a3a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4674035040 tx=0x7f4674033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f467d600640 1 -- 172.21.15.94:0/3529405932 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467403f020 con 0x7f46801008d0 2024-06-03T01:18:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f467d600640 1 -- 172.21.15.94:0/3529405932 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4674004040 con 0x7f46801008d0 2024-06-03T01:18:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f467d600640 1 -- 172.21.15.94:0/3529405932 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674037040 con 0x7f46801008d0 2024-06-03T01:18:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.177+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468019f620 con 0x7f46801008d0 2024-06-03T01:18:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.178+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46801a6f10 con 0x7f46801008d0 2024-06-03T01:18:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f467d600640 1 -- 172.21.15.94:0/3529405932 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4674042020 con 0x7f46801008d0 2024-06-03T01:18:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 Processor -- start 2024-06-03T01:18:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 -- start start 2024-06-03T01:18:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dca5b0 0x558a58dc8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dc9040 0x558a58dc94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a58d2d420 con 0x558a58dd3dc0 2024-06-03T01:18:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558a58cd4c50 con 0x558a58dca5b0 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.181+0000 7f468a9f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558a58d293e0 con 0x558a58dc9040 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dc9040 0x558a58dc94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dca5b0 0x558a58dc8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dca5b0 0x558a58dc8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:38248/0 (socket says 172.21.15.94:38248) 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685c00640 1 -- 172.21.15.94:0/3723949707 learned_addr learned my addr 172.21.15.94:0/3723949707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f467fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685200640 1 -- 172.21.15.94:0/3723949707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dca5b0 msgr2=0x558a58dc8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685200640 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dca5b0 0x558a58dc8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685200640 1 -- 172.21.15.94:0/3723949707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 msgr2=0x558a58dd61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685200640 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685200640 1 -- 172.21.15.94:0/3723949707 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a58a41210 con 0x558a58dc9040 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f4685c00640 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dca5b0 0x558a58dc8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f467fe00640 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.182+0000 7f467d600640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f46680a2da0 0x7f46680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.183+0000 7f467d600640 1 -- 172.21.15.94:0/3529405932 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7f46740ef7a0 con 0x7f46801008d0 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.183+0000 7f4685c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f46680a2da0 0x7f46680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.183+0000 7f4685200640 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dc9040 0x558a58dc94c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46740ecbb0 tx=0x7f46740f5a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9084fcb7d3fa26f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.183+0000 7f4672000640 1 -- 172.21.15.94:0/3723949707 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740b7c50 con 0x558a58dc9040 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.183+0000 7f4672000640 1 -- 172.21.15.94:0/3723949707 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46740bc0b0 con 0x558a58dc9040 2024-06-03T01:18:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.183+0000 7f4672000640 1 -- 172.21.15.94:0/3723949707 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740bb310 con 0x558a58dc9040 2024-06-03T01:18:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 -- 172.21.15.94:0/3723949707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dc9040 msgr2=0x558a58dc94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f4685c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f46680a2da0 0x7f46680a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4678011880 tx=0x7f4678009500 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dc9040 0x558a58dc94c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46740ecbb0 tx=0x7f46740f5a60 comp rx=0 tx=0).stop 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 -- 172.21.15.94:0/3723949707 shutdown_connections 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dc9040 0x558a58dc94c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3723949707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dca5b0 0x558a58dc8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 -- 172.21.15.94:0/3723949707 >> 172.21.15.94:0/3723949707 conn(0x558a58d91f80 msgr2=0x558a58dc5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 -- 172.21.15.94:0/3723949707 shutdown_connections 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 -- 172.21.15.94:0/3723949707 wait complete. 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.184+0000 7f468a9f9740 1 -- start start 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 1 Processor -- start 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dc9040 0x558a58dd0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58dd1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58dd1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558a58dd68b0 con 0x558a58dc9040 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558a58ce80b0 con 0x558a58dd3dc0 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558a58dd6a30 con 0x558a58dca5b0 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f468a9f9740 10 client.0 _ll_register_callbacks cb 0x558a58cd3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f4685200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dc9040 0x558a58dd0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f4685200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dc9040 0x558a58dd0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50754/0 (socket says 172.21.15.94:50754) 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f4685200640 1 -- 172.21.15.94:0/4142535950 learned_addr learned my addr 172.21.15.94:0/4142535950 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f4685c00640 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58dd1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.185+0000 7f467fe00640 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58dd1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4685200640 1 -- 172.21.15.94:0/4142535950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 msgr2=0x558a58dd1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4685200640 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58dd1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4685200640 1 -- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 msgr2=0x558a58dd1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4685200640 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58dd1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4685200640 1 -- 172.21.15.94:0/4142535950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a58b0d620 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4685200640 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dc9040 0x558a58dd0de0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f46740bb940 tx=0x7f4674004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ede58b173a93b98 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4666a00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4666a00640 1 -- 172.21.15.94:0/4142535950 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740bb4f0 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4666a00640 1 -- 172.21.15.94:0/4142535950 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7f46740ef320 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46740bc0b0 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f4666a00640 1 -- 172.21.15.94:0/4142535950 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f46740d9050 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.186+0000 7f468a9f9740 10 client.15816 fetch_fsmap learned FSMap version 41 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558a58dd68b0 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f4666a00640 1 -- 172.21.15.94:0/4142535950 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46740bc0b0 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f4666a00640 1 -- 172.21.15.94:0/4142535950 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f46740bcce0 con 0x558a58dc9040 2024-06-03T01:18:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f468a9f9740 10 client.15816 fetch_fsmap finished waiting for FSMap version 41 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f468a9f9740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f468a9f9740 20 client.15816 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58de3ff0 0x558a58de6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f468a9f9740 4 client.15816 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.187+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a58d3ab50 con 0x558a58de3ff0 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.188+0000 7f467fe00640 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58de3ff0 0x558a58de6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.188+0000 7f467fe00640 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58de3ff0 0x558a58de6410 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab581f439f9223b server_cookie=653cad67d047c681 in_seq=0 out_seq=0 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.188+0000 7f4666a00640 10 client.15816 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f4666a00640 1 -- 172.21.15.94:0/4142535950 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a58d3ab50 con 0x558a58de3ff0 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f4666a00640 10 client.15816 handle_command_reply: tid=0 2024-06-03T01:18:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 client.15816 shutdown 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dc9040 msgr2=0x558a58dd0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dc9040 0x558a58dd0de0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f46740bb940 tx=0x7f4674004420 comp rx=0 tx=0).stop 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 shutdown_connections 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58de3ff0 0x558a58de6410 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58dd1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58dd1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f4666a00640 0 client.15816 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4142535950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dc9040 0x558a58dd0de0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.189+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 >> 172.21.15.94:0/4142535950 conn(0x558a58d91f80 msgr2=0x558a58dcb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.190+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 shutdown_connections 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.190+0000 7f468a9f9740 1 -- 172.21.15.94:0/4142535950 wait complete. 2024-06-03T01:18:20.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.190+0000 7f468a9f9740 20 client.15816 trim_cache size 0 max 16384 2024-06-03T01:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.206+0000 7f468a9f9740 1 Processor -- start 2024-06-03T01:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.206+0000 7f468a9f9740 1 -- start start 2024-06-03T01:18:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.206+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dca5b0 0x558a58dd16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58debc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dd3620 0x558a58dec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f468a9f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46801b0f20 con 0x558a58dca5b0 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f468a9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46801b2b80 con 0x558a58dd3dc0 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f468a9f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46801b1650 con 0x558a58dd3620 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dca5b0 0x558a58dd16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dca5b0 0x558a58dd16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50762/0 (socket says 172.21.15.94:50762) 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 -- 172.21.15.94:0/3087695306 learned_addr learned my addr 172.21.15.94:0/3087695306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f467fe00640 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58debc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685c00640 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dd3620 0x558a58dec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 -- 172.21.15.94:0/3087695306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dd3620 msgr2=0x558a58dec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dd3620 0x558a58dec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 -- 172.21.15.94:0/3087695306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 msgr2=0x558a58debc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58debc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f4685200640 1 -- 172.21.15.94:0/3087695306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a58dc9b10 con 0x558a58dca5b0 2024-06-03T01:18:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.207+0000 7f467fe00640 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58debc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.208+0000 7f4685200640 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dca5b0 0x558a58dd16b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f46740b8330 tx=0x7f46740f5a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d304db727bdca53 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.208+0000 7f4671600640 1 -- 172.21.15.94:0/3087695306 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740bb780 con 0x558a58dca5b0 2024-06-03T01:18:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.208+0000 7f4671600640 1 -- 172.21.15.94:0/3087695306 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46740f5300 con 0x558a58dca5b0 2024-06-03T01:18:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.208+0000 7f4671600640 1 -- 172.21.15.94:0/3087695306 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740bb310 con 0x558a58dca5b0 2024-06-03T01:18:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 -- 172.21.15.94:0/3087695306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dca5b0 msgr2=0x558a58dd16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dca5b0 0x558a58dd16b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f46740b8330 tx=0x7f46740f5a60 comp rx=0 tx=0).stop 2024-06-03T01:18:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 -- 172.21.15.94:0/3087695306 shutdown_connections 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dd3620 0x558a58dec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3dc0 0x558a58debc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 --2- 172.21.15.94:0/3087695306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dca5b0 0x558a58dd16b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 -- 172.21.15.94:0/3087695306 >> 172.21.15.94:0/3087695306 conn(0x558a58d91f80 msgr2=0x558a58dcfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 -- 172.21.15.94:0/3087695306 shutdown_connections 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 -- 172.21.15.94:0/3087695306 wait complete. 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 -- start start 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.209+0000 7f468a9f9740 1 Processor -- start 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58ddc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3620 0x558a58dd74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f468a9f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f468a9f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46801b23f0 con 0x558a58dd3dc0 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f468a9f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46801b1d20 con 0x558a58dd3620 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f468a9f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46801aef20 con 0x558a58dca5b0 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f468a9f9740 10 client.0 _ll_register_callbacks cb 0x7f466c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f4685c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f467fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3620 0x558a58dd74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f4685c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50776/0 (socket says 172.21.15.94:50776) 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f4685c00640 1 -- 172.21.15.94:0/4191345214 learned_addr learned my addr 172.21.15.94:0/4191345214 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f4685200640 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58ddc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f467fe00640 1 -- 172.21.15.94:0/4191345214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 msgr2=0x558a58ddc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f467fe00640 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58ddc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f467fe00640 1 -- 172.21.15.94:0/4191345214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 msgr2=0x558a58dd7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f467fe00640 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f467fe00640 1 -- 172.21.15.94:0/4191345214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a58dd7f60 con 0x558a58dd3620 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.210+0000 7f4685200640 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58ddc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f467fe00640 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3620 0x558a58dd74e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x558a58dc67c0 tx=0x7f466c010a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d5744b80390701 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f4666a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f4666a00640 1 -- 172.21.15.94:0/4191345214 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c004220 con 0x558a58dd3620 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f4666a00640 1 -- 172.21.15.94:0/4191345214 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7f466c0043c0 con 0x558a58dd3620 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46740f5300 con 0x558a58dd3620 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f4666a00640 1 -- 172.21.15.94:0/4191345214 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f466c005020 con 0x558a58dd3620 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f468a9f9740 10 client.25937 fetch_fsmap learned FSMap version 41 2024-06-03T01:18:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.211+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558a58e4e9e0 con 0x558a58dd3620 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f4666a00640 1 -- 172.21.15.94:0/4191345214 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46740f5300 con 0x558a58dd3620 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f4666a00640 1 -- 172.21.15.94:0/4191345214 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f466c0047a0 con 0x558a58dd3620 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f468a9f9740 10 client.25937 fetch_fsmap finished waiting for FSMap version 41 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f468a9f9740 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f468a9f9740 20 client.25937 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58e4f220 0x558a58e51640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f468a9f9740 4 client.25937 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a58d34ea0 con 0x558a58e4f220 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.212+0000 7f4685200640 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58e4f220 0x558a58e51640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.213+0000 7f4685200640 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58e4f220 0x558a58e51640 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99e511beb4835ce server_cookie=d836fe5824103d5b in_seq=0 out_seq=0 2024-06-03T01:18:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.213+0000 7f4666a00640 10 client.25937 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.240+0000 7f4666a00640 1 -- 172.21.15.94:0/4191345214 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558a58d34ea0 con 0x558a58e4f220 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.240+0000 7f4666a00640 10 client.25937 handle_command_reply: tid=0 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.240+0000 7f468a9f9740 1 client.25937 shutdown 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3620 msgr2=0x558a58dd74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3620 0x558a58dd74e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x558a58dc67c0 tx=0x7f466c010a80 comp rx=0 tx=0).stop 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 shutdown_connections 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558a58dd3dc0 0x558a58dd7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558a58e4f220 0x558a58e51640 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a58dd3620 0x558a58dd74e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 --2- 172.21.15.94:0/4191345214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a58dca5b0 0x558a58ddc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f4666a00640 0 client.25937 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 >> 172.21.15.94:0/4191345214 conn(0x558a58d91f80 msgr2=0x558a58dd2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 shutdown_connections 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 1 -- 172.21.15.94:0/4191345214 wait complete. 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.241+0000 7f468a9f9740 20 client.25937 trim_cache size 0 max 16384 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f46680a2da0 msgr2=0x7f46680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f46680a2da0 0x7f46680a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4678011880 tx=0x7f4678009500 comp rx=0 tx=0).stop 2024-06-03T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46801008d0 msgr2=0x7f468019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46801008d0 0x7f468019a3a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4674035040 tx=0x7f4674033b60 comp rx=0 tx=0).stop 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 shutdown_connections 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f46680a2da0 0x7f46680a5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4680106890 0x7f468019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4680101290 0x7f468019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 --2- 172.21.15.94:0/3529405932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46801008d0 0x7f468019a3a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 >> 172.21.15.94:0/3529405932 conn(0x7f4680077a20 msgr2=0x7f4680069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 shutdown_connections 2024-06-03T01:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:20.242+0000 7f4687c00640 1 -- 172.21.15.94:0/3529405932 wait complete. 2024-06-03T01:18:20.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:20 smithi107 ceph-mon[36622]: pgmap v246: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-03T01:18:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:20 smithi094 ceph-mon[32119]: pgmap v246: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-03T01:18:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:20 smithi179 ceph-mon[37041]: pgmap v246: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-03T01:18:21.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 187224 2.26 MB/sec execute 268 sec latency 1600.876 ms 2024-06-03T01:18:22.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188242 2.28 MB/sec execute 269 sec latency 21.818 ms 2024-06-03T01:18:22.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:22 smithi107 ceph-mon[36622]: pgmap v247: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 909 KiB/s wr, 105 op/s 2024-06-03T01:18:22.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:22 smithi094 ceph-mon[32119]: pgmap v247: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 909 KiB/s wr, 105 op/s 2024-06-03T01:18:22.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:22 smithi179 ceph-mon[37041]: pgmap v247: 129 pgs: 129 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 909 KiB/s wr, 105 op/s 2024-06-03T01:18:23.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188727 2.27 MB/sec execute 270 sec latency 23.520 ms 2024-06-03T01:18:24.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188728 2.26 MB/sec execute 271 sec latency 986.223 ms 2024-06-03T01:18:24.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:24 smithi107 ceph-mon[36622]: pgmap v248: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-06-03T01:18:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:24 smithi094 ceph-mon[32119]: pgmap v248: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-06-03T01:18:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:24 smithi179 ceph-mon[37041]: pgmap v248: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-06-03T01:18:25.107 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188728 2.26 MB/sec execute 272 sec latency 1986.315 ms 2024-06-03T01:18:26.106 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188750 2.25 MB/sec execute 273 sec latency 2944.449 ms 2024-06-03T01:18:26.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:26 smithi094 ceph-mon[32119]: pgmap v249: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 825 KiB/s wr, 96 op/s 2024-06-03T01:18:26.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:26 smithi179 ceph-mon[37041]: pgmap v249: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 825 KiB/s wr, 96 op/s 2024-06-03T01:18:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:26 smithi107 ceph-mon[36622]: pgmap v249: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 825 KiB/s wr, 96 op/s 2024-06-03T01:18:27.107 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188796 2.24 MB/sec execute 274 sec latency 949.932 ms 2024-06-03T01:18:28.107 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188796 2.23 MB/sec execute 275 sec latency 1950.047 ms 2024-06-03T01:18:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:28 smithi094 ceph-mon[32119]: pgmap v250: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-03T01:18:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:28 smithi179 ceph-mon[37041]: pgmap v250: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-03T01:18:28.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:28 smithi107 ceph-mon[36622]: pgmap v250: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-03T01:18:29.107 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188796 2.22 MB/sec execute 276 sec latency 2950.210 ms 2024-06-03T01:18:30.107 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188796 2.22 MB/sec execute 277 sec latency 3950.302 ms 2024-06-03T01:18:30.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:30 smithi094 ceph-mon[32119]: pgmap v251: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-06-03T01:18:30.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:30 smithi179 ceph-mon[37041]: pgmap v251: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-06-03T01:18:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:30 smithi107 ceph-mon[36622]: pgmap v251: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-06-03T01:18:31.107 INFO:tasks.workunit.client.0.smithi094.stdout: 1 188818 2.21 MB/sec execute 278 sec latency 4902.086 ms 2024-06-03T01:18:32.108 INFO:tasks.workunit.client.0.smithi094.stdout: 1 189698 2.22 MB/sec execute 279 sec latency 59.790 ms 2024-06-03T01:18:32.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:32 smithi094 ceph-mon[32119]: pgmap v252: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-06-03T01:18:32.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:32 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:18:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:32 smithi179 ceph-mon[37041]: pgmap v252: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-06-03T01:18:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:32 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:18:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:32 smithi107 ceph-mon[36622]: pgmap v252: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-06-03T01:18:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:32 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:18:33.108 INFO:tasks.workunit.client.0.smithi094.stdout: 1 189698 2.21 MB/sec execute 280 sec latency 1059.929 ms 2024-06-03T01:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.383+0000 7fad13800640 1 -- 172.21.15.94:0/2225319526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c1047c0 msgr2=0x7fad0c108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.383+0000 7fad13800640 1 --2- 172.21.15.94:0/2225319526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c1047c0 0x7fad0c108800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7facf4009900 tx=0x7facf402a3f0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.384+0000 7fad13800640 1 -- 172.21.15.94:0/2225319526 shutdown_connections 2024-06-03T01:18:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.384+0000 7fad13800640 1 --2- 172.21.15.94:0/2225319526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad0c108ee0 0x7fad0c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.384+0000 7fad13800640 1 --2- 172.21.15.94:0/2225319526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c1047c0 0x7fad0c108800 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.384+0000 7fad13800640 1 --2- 172.21.15.94:0/2225319526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 0x7fad0c1041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.384+0000 7fad13800640 1 -- 172.21.15.94:0/2225319526 >> 172.21.15.94:0/2225319526 conn(0x7fad0c0ff6d0 msgr2=0x7fad0c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.384+0000 7fad13800640 1 -- 172.21.15.94:0/2225319526 shutdown_connections 2024-06-03T01:18:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.384+0000 7fad13800640 1 -- 172.21.15.94:0/2225319526 wait complete. 2024-06-03T01:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.385+0000 7fad13800640 1 Processor -- start 2024-06-03T01:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.385+0000 7fad13800640 1 -- start start 2024-06-03T01:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad13800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 0x7fad0c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad13800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad0c108ee0 0x7fad0c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad13800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c19ecd0 0x7fad0c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad13800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c113f90 con 0x7fad0c19ecd0 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad13800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c113e10 con 0x7fad0c103e10 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad13800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c114110 con 0x7fad0c108ee0 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad10e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 0x7fad0c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad10e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 0x7fad0c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37824/0 (socket says 172.21.15.94:37824) 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad10e00640 1 -- 172.21.15.94:0/1877062635 learned_addr learned my addr 172.21.15.94:0/1877062635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad11800640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c19ecd0 0x7fad0c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.386+0000 7fad03e00640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad0c108ee0 0x7fad0c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad10e00640 1 -- 172.21.15.94:0/1877062635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad0c108ee0 msgr2=0x7fad0c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad10e00640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad0c108ee0 0x7fad0c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad10e00640 1 -- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c19ecd0 msgr2=0x7fad0c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad10e00640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c19ecd0 0x7fad0c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad10e00640 1 -- 172.21.15.94:0/1877062635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c19f7c0 con 0x7fad0c103e10 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad10e00640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 0x7fad0c19a410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7facfc00e9f0 tx=0x7facfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad0a000640 1 -- 172.21.15.94:0/1877062635 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00cd80 con 0x7fad0c103e10 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad0a000640 1 -- 172.21.15.94:0/1877062635 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facfc010910 con 0x7fad0c103e10 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c19fab0 con 0x7fad0c103e10 2024-06-03T01:18:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.387+0000 7fad0a000640 1 -- 172.21.15.94:0/1877062635 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc018990 con 0x7fad0c103e10 2024-06-03T01:18:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.388+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c1a7210 con 0x7fad0c103e10 2024-06-03T01:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 Processor -- start 2024-06-03T01:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad0a000640 1 -- 172.21.15.94:0/1877062635 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7facfc010ac0 con 0x7fad0c103e10 2024-06-03T01:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 -- start start 2024-06-03T01:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563e19225420 con 0x563e192c25b0 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563e191ccc50 con 0x563e192c1040 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.391+0000 7fad1660e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563e192213e0 con 0x563e192cbdc0 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44818/0 (socket says 172.21.15.94:44818) 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 -- 172.21.15.94:0/2353983037 learned_addr learned my addr 172.21.15.94:0/2353983037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad10e00640 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad03e00640 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 -- 172.21.15.94:0/2353983037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 msgr2=0x563e192ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 -- 172.21.15.94:0/2353983037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 msgr2=0x563e192c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192c14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 -- 172.21.15.94:0/2353983037 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e18f39210 con 0x563e192c25b0 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad11800640 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c0b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fad04009f50 tx=0x7fad040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d0c3b3227e895d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.392+0000 7fad0a000640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7facf00a2da0 0x7facf00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad02000640 1 -- 172.21.15.94:0/2353983037 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04004d20 con 0x563e192c25b0 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad0a000640 1 -- 172.21.15.94:0/1877062635 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7facfc0cfb60 con 0x7fad0c103e10 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad02000640 1 -- 172.21.15.94:0/2353983037 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04004050 con 0x563e192c25b0 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad02000640 1 -- 172.21.15.94:0/2353983037 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04005280 con 0x563e192c25b0 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad10e00640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7facf00a2da0 0x7facf00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad1660e740 1 -- 172.21.15.94:0/2353983037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 msgr2=0x563e192c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad1660e740 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c0b00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fad04009f50 tx=0x7fad040118d0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.393+0000 7fad10e00640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7facf00a2da0 0x7facf00a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x563e192c3d10 tx=0x7facfc0d3610 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 -- 172.21.15.94:0/2353983037 shutdown_connections 2024-06-03T01:18:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 --2- 172.21.15.94:0/2353983037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c0b00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 -- 172.21.15.94:0/2353983037 >> 172.21.15.94:0/2353983037 conn(0x563e19289f80 msgr2=0x563e192bd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 -- 172.21.15.94:0/2353983037 shutdown_connections 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 -- 172.21.15.94:0/2353983037 wait complete. 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 -- start start 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.394+0000 7fad1660e740 1 Processor -- start 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad1660e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad1660e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad1660e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad1660e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563e192ceb90 con 0x563e192c1040 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad1660e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563e191e00b0 con 0x563e192c25b0 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad1660e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563e192ced10 con 0x563e192cbdc0 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad1660e740 10 client.0 _ll_register_callbacks cb 0x563e191cbf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad11800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37838/0 (socket says 172.21.15.94:37838) 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 -- 172.21.15.94:0/1735655265 learned_addr learned my addr 172.21.15.94:0/1735655265 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad11800640 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 -- 172.21.15.94:0/1735655265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 msgr2=0x563e192c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 -- 172.21.15.94:0/1735655265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 msgr2=0x563e192c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.395+0000 7fad03e00640 1 -- 172.21.15.94:0/1735655265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e19005620 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7fad03e00640 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192c92a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facf4009230 tx=0x7facf40317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5309ae0df52497 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7facfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7facfaa00640 1 -- 172.21.15.94:0/1735655265 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4035040 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7facfaa00640 1 -- 172.21.15.94:0/1735655265 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7facf4004240 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad04004050 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7facfaa00640 1 -- 172.21.15.94:0/1735655265 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7facf404f050 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7fad1660e740 10 client.25949 fetch_fsmap learned FSMap version 42 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.396+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e192dae90 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7facfaa00640 1 -- 172.21.15.94:0/1735655265 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04004050 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7facfaa00640 1 -- 172.21.15.94:0/1735655265 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7facf4046480 con 0x563e192c25b0 2024-06-03T01:18:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7fad1660e740 10 client.25949 fetch_fsmap finished waiting for FSMap version 42 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7fad1660e740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7fad1660e740 20 client.25949 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7fad1660e740 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e192dc410 0x563e192de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7fad1660e740 4 client.25949 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e19232b50 con 0x563e192dc410 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.397+0000 7fad11800640 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e192dc410 0x563e192de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.398+0000 7fad11800640 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e192dc410 0x563e192de800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c94aadf766e8e9 server_cookie=41ba709ee786f355 in_seq=0 out_seq=0 2024-06-03T01:18:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.398+0000 7facfaa00640 10 client.25949 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7facfaa00640 1 -- 172.21.15.94:0/1735655265 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563e19232b50 con 0x563e192dc410 2024-06-03T01:18:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7facfaa00640 10 client.25949 handle_command_reply: tid=0 2024-06-03T01:18:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 client.25949 shutdown 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 msgr2=0x563e192c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192c92a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facf4009230 tx=0x7facf40317e0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 shutdown_connections 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e192dc410 0x563e192de800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192c92a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 --2- 172.21.15.94:0/1735655265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 >> 172.21.15.94:0/1735655265 conn(0x563e19289f80 msgr2=0x563e192c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.399+0000 7facfaa00640 0 client.25949 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.400+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 shutdown_connections 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.400+0000 7fad1660e740 1 -- 172.21.15.94:0/1735655265 wait complete. 2024-06-03T01:18:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.400+0000 7fad1660e740 20 client.25949 trim_cache size 0 max 16384 2024-06-03T01:18:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.416+0000 7fad1660e740 1 Processor -- start 2024-06-03T01:18:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.416+0000 7fad1660e740 1 -- start start 2024-06-03T01:18:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad1660e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad1660e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad1660e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad1660e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b1020 con 0x563e192c25b0 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad1660e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b2cb0 con 0x563e192c1040 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad1660e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b1750 con 0x563e192cbdc0 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44832/0 (socket says 172.21.15.94:44832) 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 -- 172.21.15.94:0/373184461 learned_addr learned my addr 172.21.15.94:0/373184461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad11800640 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad11800640 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 -- 172.21.15.94:0/373184461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 msgr2=0x563e192c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 -- 172.21.15.94:0/373184461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 msgr2=0x563e192bd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192bd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad03e00640 1 -- 172.21.15.94:0/373184461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e192c72b0 con 0x563e192c25b0 2024-06-03T01:18:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.417+0000 7fad11800640 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:18:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.418+0000 7fad03e00640 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c7ec0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7facf4009850 tx=0x7facf4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e7fc645337d5c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.418+0000 7fad01600640 1 -- 172.21.15.94:0/373184461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4058070 con 0x563e192c25b0 2024-06-03T01:18:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.418+0000 7fad01600640 1 -- 172.21.15.94:0/373184461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf4046c20 con 0x563e192c25b0 2024-06-03T01:18:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.418+0000 7fad01600640 1 -- 172.21.15.94:0/373184461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4035030 con 0x563e192c25b0 2024-06-03T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 -- 172.21.15.94:0/373184461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 msgr2=0x563e192c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c7ec0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7facf4009850 tx=0x7facf4009520 comp rx=0 tx=0).stop 2024-06-03T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 -- 172.21.15.94:0/373184461 shutdown_connections 2024-06-03T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c25b0 0x563e192c7ec0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 --2- 172.21.15.94:0/373184461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c1040 0x563e192bd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 -- 172.21.15.94:0/373184461 >> 172.21.15.94:0/373184461 conn(0x563e19289f80 msgr2=0x563e192c2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 -- 172.21.15.94:0/373184461 shutdown_connections 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 -- 172.21.15.94:0/373184461 wait complete. 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 -- start start 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.419+0000 7fad1660e740 1 Processor -- start 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad1660e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad1660e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad1660e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad1660e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b2520 con 0x563e192c1040 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad1660e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1b1e20 con 0x563e192c25b0 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad1660e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1af020 con 0x563e192cbdc0 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad1660e740 10 client.0 _ll_register_callbacks cb 0x7facf4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad11800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad11800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192d4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44838/0 (socket says 172.21.15.94:44838) 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad11800640 1 -- 172.21.15.94:0/817921060 learned_addr learned my addr 172.21.15.94:0/817921060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad03e00640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.420+0000 7fad11800640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad03e00640 1 -- 172.21.15.94:0/817921060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 msgr2=0x563e192cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad03e00640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad03e00640 1 -- 172.21.15.94:0/817921060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 msgr2=0x563e192d4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad03e00640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192d4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad03e00640 1 -- 172.21.15.94:0/817921060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e192cfd60 con 0x563e192c25b0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad11800640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad03e00640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192d46c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x563e192be9c0 tx=0x7facf4046730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa1e710cc0d4a8b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7facfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7facfaa00640 1 -- 172.21.15.94:0/817921060 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4061060 con 0x563e192c25b0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf4046c20 con 0x563e192c25b0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7facfaa00640 1 -- 172.21.15.94:0/817921060 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7facf4005040 con 0x563e192c25b0 2024-06-03T01:18:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7facfaa00640 1 -- 172.21.15.94:0/817921060 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7facf4068050 con 0x563e192c25b0 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.421+0000 7fad1660e740 10 client.25955 fetch_fsmap learned FSMap version 42 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e1934f380 con 0x563e192c25b0 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7facfaa00640 1 -- 172.21.15.94:0/817921060 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf4046c20 con 0x563e192c25b0 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7facfaa00640 1 -- 172.21.15.94:0/817921060 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7facf405c460 con 0x563e192c25b0 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7fad1660e740 10 client.25955 fetch_fsmap finished waiting for FSMap version 42 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7fad1660e740 10 client.25955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7fad1660e740 20 client.25955 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7fad1660e740 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e1934fa90 0x563e19351eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7fad1660e740 4 client.25955 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.422+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e1922cea0 con 0x563e1934fa90 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.423+0000 7fad11800640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e1934fa90 0x563e19351eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.423+0000 7fad11800640 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e1934fa90 0x563e19351eb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10c1e710117f62c server_cookie=5f1af3d148901736 in_seq=0 out_seq=0 2024-06-03T01:18:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.423+0000 7facfaa00640 10 client.25955 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.447+0000 7facfaa00640 1 -- 172.21.15.94:0/817921060 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563e1922cea0 con 0x563e1934fa90 2024-06-03T01:18:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7facfaa00640 10 client.25955 handle_command_reply: tid=0 2024-06-03T01:18:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 client.25955 shutdown 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 msgr2=0x563e192d46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192d46c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x563e192be9c0 tx=0x7facf4046730 comp rx=0 tx=0).stop 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 shutdown_connections 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e192cbdc0 0x563e192cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563e1934fa90 0x563e19351eb0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563e192c25b0 0x563e192d46c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 --2- 172.21.15.94:0/817921060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563e192c1040 0x563e192d4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7facfaa00640 0 client.25955 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 >> 172.21.15.94:0/817921060 conn(0x563e19289f80 msgr2=0x563e192c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 shutdown_connections 2024-06-03T01:18:33.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 1 -- 172.21.15.94:0/817921060 wait complete. 2024-06-03T01:18:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.448+0000 7fad1660e740 20 client.25955 trim_cache size 0 max 16384 2024-06-03T01:18:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:18:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:18:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7facf00a2da0 msgr2=0x7facf00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7facf00a2da0 0x7facf00a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x563e192c3d10 tx=0x7facfc0d3610 comp rx=0 tx=0).stop 2024-06-03T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 msgr2=0x7fad0c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 0x7fad0c19a410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7facfc00e9f0 tx=0x7facfc00eeb0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 shutdown_connections 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7facf00a2da0 0x7facf00a5260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad0c19ecd0 0x7fad0c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad0c108ee0 0x7fad0c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 --2- 172.21.15.94:0/1877062635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad0c103e10 0x7fad0c19a410 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 >> 172.21.15.94:0/1877062635 conn(0x7fad0c0ff6d0 msgr2=0x7fad0c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 shutdown_connections 2024-06-03T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:33.449+0000 7fad13800640 1 -- 172.21.15.94:0/1877062635 wait complete. 2024-06-03T01:18:34.108 INFO:tasks.workunit.client.0.smithi094.stdout: 1 189698 2.20 MB/sec execute 281 sec latency 2060.040 ms 2024-06-03T01:18:34.334 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:34 smithi107 ceph-mon[36622]: pgmap v253: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-03T01:18:34.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:34 smithi094 ceph-mon[32119]: pgmap v253: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-03T01:18:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:34 smithi179 ceph-mon[37041]: pgmap v253: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-06-03T01:18:35.108 INFO:tasks.workunit.client.0.smithi094.stdout: 1 189698 2.19 MB/sec execute 282 sec latency 3060.208 ms 2024-06-03T01:18:36.108 INFO:tasks.workunit.client.0.smithi094.stdout: 1 190765 2.21 MB/sec execute 283 sec latency 3279.550 ms 2024-06-03T01:18:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:36 smithi094 ceph-mon[32119]: pgmap v254: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 475 KiB/s wr, 51 op/s 2024-06-03T01:18:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:36 smithi179 ceph-mon[37041]: pgmap v254: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 475 KiB/s wr, 51 op/s 2024-06-03T01:18:36.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:36 smithi107 ceph-mon[36622]: pgmap v254: 129 pgs: 129 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 475 KiB/s wr, 51 op/s 2024-06-03T01:18:37.108 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191367 2.21 MB/sec execute 284 sec latency 491.981 ms 2024-06-03T01:18:37.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:37 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:18:37.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:37 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:18:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:37 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:18:38.109 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191367 2.20 MB/sec execute 285 sec latency 1492.732 ms 2024-06-03T01:18:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:38 smithi179 ceph-mon[37041]: pgmap v255: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 829 KiB/s wr, 91 op/s 2024-06-03T01:18:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:38 smithi107 ceph-mon[36622]: pgmap v255: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 829 KiB/s wr, 91 op/s 2024-06-03T01:18:38.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:38 smithi094 ceph-mon[32119]: pgmap v255: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 829 KiB/s wr, 91 op/s 2024-06-03T01:18:39.109 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191380 2.20 MB/sec execute 286 sec latency 1518.645 ms 2024-06-03T01:18:40.109 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191380 2.19 MB/sec execute 287 sec latency 1945.553 ms 2024-06-03T01:18:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:40 smithi179 ceph-mon[37041]: pgmap v256: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 449 KiB/s wr, 57 op/s 2024-06-03T01:18:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:40 smithi107 ceph-mon[36622]: pgmap v256: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 449 KiB/s wr, 57 op/s 2024-06-03T01:18:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:40 smithi094 ceph-mon[32119]: pgmap v256: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 449 KiB/s wr, 57 op/s 2024-06-03T01:18:41.109 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191380 2.18 MB/sec execute 288 sec latency 2945.649 ms 2024-06-03T01:18:42.109 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191380 2.17 MB/sec execute 289 sec latency 3945.755 ms 2024-06-03T01:18:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:42 smithi107 ceph-mon[36622]: pgmap v257: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 449 KiB/s wr, 57 op/s 2024-06-03T01:18:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:42 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:42 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:42 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:42 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:42 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:42 smithi094 ceph-mon[32119]: pgmap v257: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 449 KiB/s wr, 57 op/s 2024-06-03T01:18:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:42 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:42 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:42 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:42 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:42 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:42 smithi179 ceph-mon[37041]: pgmap v257: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 449 KiB/s wr, 57 op/s 2024-06-03T01:18:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:42 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:18:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:42 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:42 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:42 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:42 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:43.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 191380 2.17 MB/sec execute 290 sec latency 4945.895 ms 2024-06-03T01:18:43.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:43 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:43.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:43 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:43.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:43 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:43 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:43 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:43.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:43 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:43 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:18:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:43 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:18:43.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:43 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:18:44.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192074 2.17 MB/sec execute 291 sec latency 4961.256 ms 2024-06-03T01:18:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:44 smithi107 ceph-mon[36622]: pgmap v258: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 829 KiB/s wr, 78 op/s 2024-06-03T01:18:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:44 smithi094 ceph-mon[32119]: pgmap v258: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 829 KiB/s wr, 78 op/s 2024-06-03T01:18:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:44 smithi179 ceph-mon[37041]: pgmap v258: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 829 KiB/s wr, 78 op/s 2024-06-03T01:18:45.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192269 2.17 MB/sec execute 292 sec latency 759.704 ms 2024-06-03T01:18:46.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192327 2.16 MB/sec execute 293 sec latency 1701.912 ms 2024-06-03T01:18:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:46 smithi107 ceph-mon[36622]: pgmap v259: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 734 KiB/s wr, 61 op/s 2024-06-03T01:18:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 -- 172.21.15.94:0/706550265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 msgr2=0x7fb100110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 --2- 172.21.15.94:0/706550265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 0x7fb100110ab0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009ca0 tx=0x7fb0f802a8c0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 -- 172.21.15.94:0/706550265 shutdown_connections 2024-06-03T01:18:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 --2- 172.21.15.94:0/706550265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 0x7fb100110ab0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 --2- 172.21.15.94:0/706550265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1001047c0 0x7fb100108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 --2- 172.21.15.94:0/706550265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 0x7fb1001041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 -- 172.21.15.94:0/706550265 >> 172.21.15.94:0/706550265 conn(0x7fb1000ff6d0 msgr2=0x7fb100101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 -- 172.21.15.94:0/706550265 shutdown_connections 2024-06-03T01:18:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.589+0000 7fb105200640 1 -- 172.21.15.94:0/706550265 wait complete. 2024-06-03T01:18:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.590+0000 7fb105200640 1 Processor -- start 2024-06-03T01:18:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.590+0000 7fb105200640 1 -- start start 2024-06-03T01:18:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.590+0000 7fb105200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 0x7fb10019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.590+0000 7fb105200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 0x7fb10019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.590+0000 7fb105200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10019eca0 0x7fb10019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.590+0000 7fb105200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb100113fe0 con 0x7fb10019eca0 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.590+0000 7fb105200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb100113e60 con 0x7fb100108ee0 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb105200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb100114160 con 0x7fb100103e10 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 0x7fb10019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 0x7fb10019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:56472/0 (socket says 172.21.15.94:56472) 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fe800640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10019eca0 0x7fb10019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 0x7fb10019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 -- 172.21.15.94:0/688596151 learned_addr learned my addr 172.21.15.94:0/688596151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 -- 172.21.15.94:0/688596151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 msgr2=0x7fb10019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 0x7fb10019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 -- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10019eca0 msgr2=0x7fb10019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10019eca0 0x7fb10019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fde00640 1 -- 172.21.15.94:0/688596151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10019f790 con 0x7fb100103e10 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 0x7fb10019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.591+0000 7fb0fe800640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10019eca0 0x7fb10019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.592+0000 7fb0fde00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 0x7fb10019a3e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e9f0 tx=0x7fb0f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.592+0000 7fb0f6a00640 1 -- 172.21.15.94:0/688596151 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000cd80 con 0x7fb100103e10 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.592+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10019fa80 con 0x7fb100103e10 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.592+0000 7fb0f6a00640 1 -- 172.21.15.94:0/688596151 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f0010910 con 0x7fb100103e10 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.592+0000 7fb0f6a00640 1 -- 172.21.15.94:0/688596151 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00188f0 con 0x7fb100103e10 2024-06-03T01:18:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.592+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1001a7260 con 0x7fb100103e10 2024-06-03T01:18:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 Processor -- start 2024-06-03T01:18:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb0f6a00640 1 -- 172.21.15.94:0/688596151 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb0f0010ac0 con 0x7fb100103e10 2024-06-03T01:18:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 -- start start 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b97040 0x55bcd1b974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bcd1afb420 con 0x55bcd1b985b0 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bcd1aa2c50 con 0x55bcd1b97040 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb107ff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bcd1af73e0 con 0x55bcd1ba1dc0 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b96b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40042/0 (socket says 172.21.15.94:40042) 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb0fe800640 1 -- 172.21.15.94:0/3112055014 learned_addr learned my addr 172.21.15.94:0/3112055014 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.596+0000 7fb0fde00640 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b97040 0x55bcd1b974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0fe800640 1 -- 172.21.15.94:0/3112055014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1ba1dc0 msgr2=0x55bcd1ba41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0fe800640 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0fe800640 1 -- 172.21.15.94:0/3112055014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b97040 msgr2=0x55bcd1b974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0fe800640 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b97040 0x55bcd1b974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0fe800640 1 -- 172.21.15.94:0/3112055014 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd180f210 con 0x55bcd1b985b0 2024-06-03T01:18:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0fe800640 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b96b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009ca0 tx=0x7fb0f8033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaca3e955c805ab0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0f6a00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb0e40a2e20 0x7fb0e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.597+0000 7fb0ef400640 1 -- 172.21.15.94:0/3112055014 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003b70 con 0x55bcd1b985b0 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb0ef400640 1 -- 172.21.15.94:0/3112055014 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8036040 con 0x55bcd1b985b0 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb0ef400640 1 -- 172.21.15.94:0/3112055014 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8004580 con 0x55bcd1b985b0 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb0f6a00640 1 -- 172.21.15.94:0/688596151 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7fb0f00cfb80 con 0x7fb100103e10 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb0fde00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb0e40a2e20 0x7fb0e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb107ff0740 1 -- 172.21.15.94:0/3112055014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 msgr2=0x55bcd1b96b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb107ff0740 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b96b00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009ca0 tx=0x7fb0f8033000 comp rx=0 tx=0).stop 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb0fde00640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb0e40a2e20 0x7fb0e40a52e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55bcd1b99d10 tx=0x7fb0f00d3630 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb107ff0740 1 -- 172.21.15.94:0/3112055014 shutdown_connections 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb107ff0740 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb107ff0740 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b97040 0x55bcd1b974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.598+0000 7fb107ff0740 1 --2- 172.21.15.94:0/3112055014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b96b00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.599+0000 7fb107ff0740 1 -- 172.21.15.94:0/3112055014 >> 172.21.15.94:0/3112055014 conn(0x55bcd1b5ff80 msgr2=0x55bcd1b93630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.599+0000 7fb107ff0740 1 -- 172.21.15.94:0/3112055014 shutdown_connections 2024-06-03T01:18:46.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:46 smithi094 ceph-mon[32119]: pgmap v259: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 734 KiB/s wr, 61 op/s 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.599+0000 7fb107ff0740 1 -- 172.21.15.94:0/3112055014 wait complete. 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.599+0000 7fb107ff0740 1 -- start start 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.599+0000 7fb107ff0740 1 Processor -- start 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1b9f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b9f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb107ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bcd1ba49b0 con 0x55bcd1ba1dc0 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb107ff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bcd1ab60b0 con 0x55bcd1b985b0 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb107ff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bcd1ba4b30 con 0x55bcd1b97040 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb107ff0740 10 client.0 _ll_register_callbacks cb 0x55bcd1aa1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b9f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb0f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1b9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:56484/0 (socket says 172.21.15.94:56484) 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb0fe800640 1 -- 172.21.15.94:0/66417367 learned_addr learned my addr 172.21.15.94:0/66417367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.600+0000 7fb0f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1b9f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57888/0 (socket says 172.21.15.94:57888) 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0f7e00640 1 -- 172.21.15.94:0/66417367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 msgr2=0x55bcd1b9edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0f7e00640 1 -- 172.21.15.94:0/66417367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 msgr2=0x55bcd1b9f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b9f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0f7e00640 1 -- 172.21.15.94:0/66417367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd18db620 con 0x55bcd1b985b0 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0fe800640 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1b9f300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55bcd1b97ae0 tx=0x7fb0e800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca16431dceceb79 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:18:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0e3e00640 1 -- 172.21.15.94:0/66417367 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8012110 con 0x55bcd1b985b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f8036040 con 0x55bcd1b985b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0e3e00640 1 -- 172.21.15.94:0/66417367 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7fb0e8004050 con 0x55bcd1b985b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.601+0000 7fb0e3e00640 1 -- 172.21.15.94:0/66417367 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb0e80122b0 con 0x55bcd1b985b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 10 client.25967 fetch_fsmap learned FSMap version 43 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bcd1bb0da0 con 0x55bcd1b985b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb0e3e00640 1 -- 172.21.15.94:0/66417367 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8036040 con 0x55bcd1b985b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb0e3e00640 1 -- 172.21.15.94:0/66417367 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fb0e8010430 con 0x55bcd1b985b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 10 client.25967 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 20 client.25967 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1bb22b0 0x55bcd1bb46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 4 client.25967 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcd1b08b50 con 0x55bcd1bb22b0 2024-06-03T01:18:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.602+0000 7fb0fe800640 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1bb22b0 0x55bcd1bb46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.603+0000 7fb0fe800640 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1bb22b0 0x55bcd1bb46d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239e12a8af3e460a server_cookie=5c22c4176c661a66 in_seq=0 out_seq=0 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.603+0000 7fb0e3e00640 10 client.25967 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb0e3e00640 1 -- 172.21.15.94:0/66417367 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bcd1b08b50 con 0x55bcd1bb22b0 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb0e3e00640 10 client.25967 handle_command_reply: tid=0 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 client.25967 shutdown 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 msgr2=0x55bcd1b9f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1b9f300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55bcd1b97ae0 tx=0x7fb0e800f790 comp rx=0 tx=0).stop 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 shutdown_connections 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b9f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1bb22b0 0x55bcd1bb46d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1b9f300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 --2- 172.21.15.94:0/66417367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb0e3e00640 0 client.25967 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 >> 172.21.15.94:0/66417367 conn(0x55bcd1b5ff80 msgr2=0x55bcd1b99430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 shutdown_connections 2024-06-03T01:18:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 1 -- 172.21.15.94:0/66417367 wait complete. 2024-06-03T01:18:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.604+0000 7fb107ff0740 20 client.25967 trim_cache size 0 max 16384 2024-06-03T01:18:46.610 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:46 smithi179 ceph-mon[37041]: pgmap v259: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 734 KiB/s wr, 61 op/s 2024-06-03T01:18:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 Processor -- start 2024-06-03T01:18:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 -- start start 2024-06-03T01:18:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b92f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001b0fb0 con 0x55bcd1b985b0 2024-06-03T01:18:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001b2ca0 con 0x55bcd1ba1dc0 2024-06-03T01:18:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.621+0000 7fb107ff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001b16e0 con 0x55bcd1b97040 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:56498/0 (socket says 172.21.15.94:56498) 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 -- 172.21.15.94:0/2653807576 learned_addr learned my addr 172.21.15.94:0/2653807576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b92f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 -- 172.21.15.94:0/2653807576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1ba1dc0 msgr2=0x55bcd1b92f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b92f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 -- 172.21.15.94:0/2653807576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 msgr2=0x55bcd1b992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 -- 172.21.15.94:0/2653807576 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd1bb4d50 con 0x55bcd1b97040 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0fe800640 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9df20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8004af0 tx=0x7fb0f80040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e2609fbee43597 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0eea00640 1 -- 172.21.15.94:0/2653807576 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003c80 con 0x55bcd1b97040 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.622+0000 7fb0eea00640 1 -- 172.21.15.94:0/2653807576 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8036070 con 0x55bcd1b97040 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb0eea00640 1 -- 172.21.15.94:0/2653807576 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f802f870 con 0x55bcd1b97040 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 -- 172.21.15.94:0/2653807576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 msgr2=0x55bcd1b9df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9df20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8004af0 tx=0x7fb0f80040e0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 -- 172.21.15.94:0/2653807576 shutdown_connections 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1b92f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1b985b0 0x55bcd1b992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2653807576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1b9df20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 -- 172.21.15.94:0/2653807576 >> 172.21.15.94:0/2653807576 conn(0x55bcd1b5ff80 msgr2=0x55bcd1b98ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 -- 172.21.15.94:0/2653807576 shutdown_connections 2024-06-03T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.623+0000 7fb107ff0740 1 -- 172.21.15.94:0/2653807576 wait complete. 2024-06-03T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 -- start start 2024-06-03T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 Processor -- start 2024-06-03T01:18:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1ba4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1ba52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001b2510 con 0x55bcd1ba1dc0 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001b1db0 con 0x55bcd1b985b0 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1001aefb0 con 0x55bcd1b97040 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.624+0000 7fb107ff0740 10 client.0 _ll_register_callbacks cb 0x7fb0e8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1ba4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0fe800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1ba4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:56500/0 (socket says 172.21.15.94:56500) 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0fe800640 1 -- 172.21.15.94:0/2930734316 learned_addr learned my addr 172.21.15.94:0/2930734316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1ba52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0fe800640 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0f7e00640 1 -- 172.21.15.94:0/2930734316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 msgr2=0x55bcd1ba4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1ba4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0f7e00640 1 -- 172.21.15.94:0/2930734316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 msgr2=0x55bcd1ba5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0f7e00640 1 -- 172.21.15.94:0/2930734316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd1c1b100 con 0x55bcd1b985b0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0f7e00640 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1ba52c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8010a40 tx=0x7fb0e8012a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=604d2ffb55e042e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.625+0000 7fb0e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb0e3e00640 1 -- 172.21.15.94:0/2930734316 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e802e070 con 0x55bcd1b985b0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f8036070 con 0x55bcd1b985b0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb0e3e00640 1 -- 172.21.15.94:0/2930734316 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7fb0e80050b0 con 0x55bcd1b985b0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb0e3e00640 1 -- 172.21.15.94:0/2930734316 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb0e8010640 con 0x55bcd1b985b0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 10 client.25973 fetch_fsmap learned FSMap version 43 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bcd1c1c470 con 0x55bcd1b985b0 2024-06-03T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb0e3e00640 1 -- 172.21.15.94:0/2930734316 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8036070 con 0x55bcd1b985b0 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb0e3e00640 1 -- 172.21.15.94:0/2930734316 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fb0e802a430 con 0x55bcd1b985b0 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 10 client.25973 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 20 client.25973 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1c1cce0 0x55bcd1c1f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 4 client.25973 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.626+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcd1b02ea0 con 0x55bcd1c1cce0 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.627+0000 7fb0fe800640 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1c1cce0 0x55bcd1c1f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.627+0000 7fb0fe800640 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1c1cce0 0x55bcd1c1f100 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5c780a622b8823 server_cookie=f91224c8c8154eb9 in_seq=0 out_seq=0 2024-06-03T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.627+0000 7fb0e3e00640 10 client.25973 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:46.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb0e3e00640 1 -- 172.21.15.94:0/2930734316 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bcd1b02ea0 con 0x55bcd1c1cce0 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb0e3e00640 10 client.25973 handle_command_reply: tid=0 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 client.25973 shutdown 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 msgr2=0x55bcd1ba52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1ba52c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8010a40 tx=0x7fb0e8012a00 comp rx=0 tx=0).stop 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 shutdown_connections 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bcd1ba1dc0 0x55bcd1ba5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bcd1c1cce0 0x55bcd1c1f100 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcd1b985b0 0x55bcd1ba52c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 --2- 172.21.15.94:0/2930734316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bcd1b97040 0x55bcd1ba4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 >> 172.21.15.94:0/2930734316 conn(0x55bcd1b5ff80 msgr2=0x55bcd1b9f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.651+0000 7fb0e3e00640 0 client.25973 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.652+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 shutdown_connections 2024-06-03T01:18:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.652+0000 7fb107ff0740 1 -- 172.21.15.94:0/2930734316 wait complete. 2024-06-03T01:18:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.652+0000 7fb107ff0740 20 client.25973 trim_cache size 0 max 16384 2024-06-03T01:18:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:18:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:18:46.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:18:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.652+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb0e40a2e20 msgr2=0x7fb0e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.652+0000 7fb105200640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb0e40a2e20 0x7fb0e40a52e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55bcd1b99d10 tx=0x7fb0f00d3630 comp rx=0 tx=0).stop 2024-06-03T01:18:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.652+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 msgr2=0x7fb10019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 0x7fb10019a3e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e9f0 tx=0x7fb0f000eeb0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 shutdown_connections 2024-06-03T01:18:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb0e40a2e20 0x7fb0e40a52e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb10019eca0 0x7fb10019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb100108ee0 0x7fb10019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 --2- 172.21.15.94:0/688596151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb100103e10 0x7fb10019a3e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 >> 172.21.15.94:0/688596151 conn(0x7fb1000ff6d0 msgr2=0x7fb100100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 shutdown_connections 2024-06-03T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:46.653+0000 7fb105200640 1 -- 172.21.15.94:0/688596151 wait complete. 2024-06-03T01:18:47.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192337 2.15 MB/sec execute 294 sec latency 980.658 ms 2024-06-03T01:18:48.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192337 2.15 MB/sec execute 295 sec latency 1980.768 ms 2024-06-03T01:18:48.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:48 smithi107 ceph-mon[36622]: pgmap v260: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 901 KiB/s wr, 95 op/s 2024-06-03T01:18:48.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:48 smithi094 ceph-mon[32119]: pgmap v260: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 901 KiB/s wr, 95 op/s 2024-06-03T01:18:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:48 smithi179 ceph-mon[37041]: pgmap v260: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 901 KiB/s wr, 95 op/s 2024-06-03T01:18:49.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192337 2.14 MB/sec execute 296 sec latency 2980.922 ms 2024-06-03T01:18:50.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192337 2.13 MB/sec execute 297 sec latency 3981.067 ms 2024-06-03T01:18:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:50 smithi107 ceph-mon[36622]: pgmap v261: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 547 KiB/s wr, 54 op/s 2024-06-03T01:18:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:50 smithi094 ceph-mon[32119]: pgmap v261: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 547 KiB/s wr, 54 op/s 2024-06-03T01:18:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:50 smithi179 ceph-mon[37041]: pgmap v261: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 547 KiB/s wr, 54 op/s 2024-06-03T01:18:51.110 INFO:tasks.workunit.client.0.smithi094.stdout: 1 192402 2.12 MB/sec execute 298 sec latency 4920.784 ms 2024-06-03T01:18:52.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 193239 2.13 MB/sec execute 299 sec latency 199.706 ms 2024-06-03T01:18:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:52 smithi107 ceph-mon[36622]: pgmap v262: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 547 KiB/s wr, 54 op/s 2024-06-03T01:18:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:52 smithi094 ceph-mon[32119]: pgmap v262: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 547 KiB/s wr, 54 op/s 2024-06-03T01:18:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:52 smithi179 ceph-mon[37041]: pgmap v262: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 547 KiB/s wr, 54 op/s 2024-06-03T01:18:53.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 193239 2.13 MB/sec execute 300 sec latency 1199.814 ms 2024-06-03T01:18:54.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 193239 2.12 MB/sec execute 301 sec latency 2199.915 ms 2024-06-03T01:18:54.406 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:54 smithi179 ceph-mon[37041]: pgmap v263: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 895 KiB/s wr, 86 op/s 2024-06-03T01:18:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:54 smithi107 ceph-mon[36622]: pgmap v263: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 895 KiB/s wr, 86 op/s 2024-06-03T01:18:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:54 smithi094 ceph-mon[32119]: pgmap v263: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 895 KiB/s wr, 86 op/s 2024-06-03T01:18:55.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 193239 2.11 MB/sec execute 302 sec latency 3200.007 ms 2024-06-03T01:18:56.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 194462 2.13 MB/sec execute 303 sec latency 3405.809 ms 2024-06-03T01:18:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:56 smithi107 ceph-mon[36622]: pgmap v264: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 515 KiB/s wr, 65 op/s 2024-06-03T01:18:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:56 smithi094 ceph-mon[32119]: pgmap v264: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 515 KiB/s wr, 65 op/s 2024-06-03T01:18:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:56 smithi179 ceph-mon[37041]: pgmap v264: 129 pgs: 129 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 515 KiB/s wr, 65 op/s 2024-06-03T01:18:57.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 195496 2.14 MB/sec execute 304 sec latency 17.094 ms 2024-06-03T01:18:58.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 195763 2.14 MB/sec execute 305 sec latency 625.321 ms 2024-06-03T01:18:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:18:58 smithi107 ceph-mon[36622]: pgmap v265: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 895 KiB/s wr, 100 op/s 2024-06-03T01:18:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:18:58 smithi094 ceph-mon[32119]: pgmap v265: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 895 KiB/s wr, 100 op/s 2024-06-03T01:18:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:18:58 smithi179 ceph-mon[37041]: pgmap v265: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 895 KiB/s wr, 100 op/s 2024-06-03T01:18:59.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 195763 2.13 MB/sec execute 306 sec latency 1625.486 ms 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 -- 172.21.15.94:0/3798689709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955410de70 msgr2=0x7f955410e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 --2- 172.21.15.94:0/3798689709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955410de70 0x7f955410e250 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9544009890 tx=0x7f954402a360 comp rx=0 tx=0).stop 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 -- 172.21.15.94:0/3798689709 shutdown_connections 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 --2- 172.21.15.94:0/3798689709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9554108860 0x7f9554114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 --2- 172.21.15.94:0/3798689709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9554107ee0 0x7f9554108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 --2- 172.21.15.94:0/3798689709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955410de70 0x7f955410e250 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 -- 172.21.15.94:0/3798689709 >> 172.21.15.94:0/3798689709 conn(0x7f9554077780 msgr2=0x7f95541067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 -- 172.21.15.94:0/3798689709 shutdown_connections 2024-06-03T01:18:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.794+0000 7f9559000640 1 -- 172.21.15.94:0/3798689709 wait complete. 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.795+0000 7f9559000640 1 Processor -- start 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.795+0000 7f9559000640 1 -- start start 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9559000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9554107ee0 0x7f955419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9559000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554108860 0x7f955419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9559000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f955410de70 0x7f95541a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9559000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9554118190 con 0x7f9554107ee0 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9559000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9554118010 con 0x7f955410de70 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9559000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9554118310 con 0x7f9554108860 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9553e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f955410de70 0x7f95541a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9553400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9554107ee0 0x7f955419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9553e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f955410de70 0x7f95541a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:59622/0 (socket says 172.21.15.94:59622) 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9552a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554108860 0x7f955419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.796+0000 7f9553e00640 1 -- 172.21.15.94:0/951462890 learned_addr learned my addr 172.21.15.94:0/951462890 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9553e00640 1 -- 172.21.15.94:0/951462890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554108860 msgr2=0x7f955419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9553e00640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554108860 0x7f955419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9553e00640 1 -- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9554107ee0 msgr2=0x7f955419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9553e00640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9554107ee0 0x7f955419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9553e00640 1 -- 172.21.15.94:0/951462890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95541a36c0 con 0x7f955410de70 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9553400640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9554107ee0 0x7f955419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9553e00640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f955410de70 0x7f95541a2f90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f954c00efc0 tx=0x7f954c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f954be00640 1 -- 172.21.15.94:0/951462890 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c003970 con 0x7f955410de70 2024-06-03T01:18:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95541a39b0 con 0x7f955410de70 2024-06-03T01:18:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f954be00640 1 -- 172.21.15.94:0/951462890 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954c004020 con 0x7f955410de70 2024-06-03T01:18:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f954be00640 1 -- 172.21.15.94:0/951462890 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c00b9f0 con 0x7f955410de70 2024-06-03T01:18:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.797+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95541ab240 con 0x7f955410de70 2024-06-03T01:18:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.801+0000 7f955d42a740 1 Processor -- start 2024-06-03T01:18:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f954be00640 1 -- 172.21.15.94:0/951462890 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f954c01d020 con 0x7f955410de70 2024-06-03T01:18:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f955d42a740 1 -- start start 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f955d42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa81cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f955d42a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f955d42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa8cf80 0x55b54aa8f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f955d42a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b54a9e6420 con 0x55b54aa82200 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f955d42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b54a98dc50 con 0x55b54aa8cf80 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.802+0000 7f955d42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b54a9e23e0 con 0x55b54aa83770 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9553400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9552a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9553400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa82680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60904/0 (socket says 172.21.15.94:60904) 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9553400640 1 -- 172.21.15.94:0/1206543994 learned_addr learned my addr 172.21.15.94:0/1206543994 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9552a00640 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa8cf80 0x55b54aa8f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9552a00640 1 -- 172.21.15.94:0/1206543994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa8cf80 msgr2=0x55b54aa8f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9552a00640 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa8cf80 0x55b54aa8f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9552a00640 1 -- 172.21.15.94:0/1206543994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 msgr2=0x55b54aa82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9552a00640 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa82680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9552a00640 1 -- 172.21.15.94:0/1206543994 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54a6fa210 con 0x55b54aa83770 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.803+0000 7f9553400640 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.804+0000 7f9552a00640 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa81cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9540011740 tx=0x7f9540004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ed83d1d71e50e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.804+0000 7f954be00640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95380a2d70 0x7f95380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.804+0000 7f9548c00640 1 -- 172.21.15.94:0/1206543994 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540004050 con 0x55b54aa83770 2024-06-03T01:18:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.804+0000 7f954be00640 1 -- 172.21.15.94:0/951462890 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7f954c0cf5f0 con 0x7f955410de70 2024-06-03T01:18:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.804+0000 7f9548c00640 1 -- 172.21.15.94:0/1206543994 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954000fca0 con 0x55b54aa83770 2024-06-03T01:18:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.804+0000 7f9548c00640 1 -- 172.21.15.94:0/1206543994 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540003970 con 0x55b54aa83770 2024-06-03T01:18:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.804+0000 7f9553400640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95380a2d70 0x7f95380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 -- 172.21.15.94:0/1206543994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 msgr2=0x55b54aa81cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa81cc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9540011740 tx=0x7f9540004730 comp rx=0 tx=0).stop 2024-06-03T01:18:59.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 -- 172.21.15.94:0/1206543994 shutdown_connections 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa8cf80 0x55b54aa8f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 --2- 172.21.15.94:0/1206543994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa81cc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 -- 172.21.15.94:0/1206543994 >> 172.21.15.94:0/1206543994 conn(0x55b54aa4af80 msgr2=0x55b54aa7e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f9553400640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95380a2d70 0x7f95380a5230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9544000c00 tx=0x7f95440056d0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 -- 172.21.15.94:0/1206543994 shutdown_connections 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 -- 172.21.15.94:0/1206543994 wait complete. 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.805+0000 7f955d42a740 1 -- start start 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 1 Processor -- start 2024-06-03T01:18:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa89f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa83770 0x55b54aa8a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aa8a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b54aa8fd50 con 0x55b54aa83770 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b54a9a10b0 con 0x55b54aa82200 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b54aa8fed0 con 0x55b54aa8cf80 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f955d42a740 10 client.0 _ll_register_callbacks cb 0x55b54a98cf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f9552a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa83770 0x55b54aa8a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f9552a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa83770 0x55b54aa8a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60908/0 (socket says 172.21.15.94:60908) 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f9552a00640 1 -- 172.21.15.94:0/2280705537 learned_addr learned my addr 172.21.15.94:0/2280705537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f9553e00640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aa8a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.806+0000 7f9553400640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa89f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f9553e00640 1 -- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 msgr2=0x55b54aa89f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f9553e00640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa89f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f9553e00640 1 -- 172.21.15.94:0/2280705537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa83770 msgr2=0x55b54aa8a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f9553e00640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa83770 0x55b54aa8a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f9553e00640 1 -- 172.21.15.94:0/2280705537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54a7c6620 con 0x55b54aa8cf80 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f9552a00640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa83770 0x55b54aa8a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f9553e00640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aa8a9e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f954c0cc8f0 tx=0x7f954c004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c58375835d35c58 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f953d600640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f953d600640 1 -- 172.21.15.94:0/2280705537 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c09faa0 con 0x55b54aa8cf80 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f953d600640 1 -- 172.21.15.94:0/2280705537 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7f954c0b4670 con 0x55b54aa8cf80 2024-06-03T01:18:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.807+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954000fca0 con 0x55b54aa8cf80 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f953d600640 1 -- 172.21.15.94:0/2280705537 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f954c0a4360 con 0x55b54aa8cf80 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 10 client.26002 fetch_fsmap learned FSMap version 43 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b54aa9c110 con 0x55b54aa8cf80 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f953d600640 1 -- 172.21.15.94:0/2280705537 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f954000fca0 con 0x55b54aa8cf80 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f953d600640 1 -- 172.21.15.94:0/2280705537 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f954c00b040 con 0x55b54aa8cf80 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 10 client.26002 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 20 client.26002 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54aa9d640 0x55b54aa9fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 4 client.26002 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.808+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b54a9f3b50 con 0x55b54aa9d640 2024-06-03T01:18:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.809+0000 7f9553400640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54aa9d640 0x55b54aa9fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.809+0000 7f9553400640 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54aa9d640 0x55b54aa9fa50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4811065e8b2f8356 server_cookie=8cec74e9bc2fa417 in_seq=0 out_seq=0 2024-06-03T01:18:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.809+0000 7f953d600640 10 client.26002 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f953d600640 1 -- 172.21.15.94:0/2280705537 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b54a9f3b50 con 0x55b54aa9d640 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f953d600640 10 client.26002 handle_command_reply: tid=0 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 client.26002 shutdown 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 msgr2=0x55b54aa8a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aa8a9e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f954c0cc8f0 tx=0x7f954c004290 comp rx=0 tx=0).stop 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 shutdown_connections 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aa8a9e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa83770 0x55b54aa8a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54aa9d640 0x55b54aa9fa50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 --2- 172.21.15.94:0/2280705537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f953d600640 0 client.26002 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.810+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 >> 172.21.15.94:0/2280705537 conn(0x55b54aa4af80 msgr2=0x55b54aa845f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.811+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 shutdown_connections 2024-06-03T01:18:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.811+0000 7f955d42a740 1 -- 172.21.15.94:0/2280705537 wait complete. 2024-06-03T01:18:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.811+0000 7f955d42a740 20 client.26002 trim_cache size 0 max 16384 2024-06-03T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.827+0000 7f955d42a740 1 Processor -- start 2024-06-03T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.827+0000 7f955d42a740 1 -- start start 2024-06-03T01:18:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f955d42a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa84470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f955d42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa83770 0x55b54aaa5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f955d42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aaa58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f955d42a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95541b5270 con 0x55b54aa82200 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f955d42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95541b6f60 con 0x55b54aa83770 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f955d42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f95541b59a0 con 0x55b54aa8cf80 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa84470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa84470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60916/0 (socket says 172.21.15.94:60916) 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 -- 172.21.15.94:0/3950052580 learned_addr learned my addr 172.21.15.94:0/3950052580 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9552a00640 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa83770 0x55b54aaa5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553e00640 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aaa58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 -- 172.21.15.94:0/3950052580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 msgr2=0x55b54aaa58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aaa58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 -- 172.21.15.94:0/3950052580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa83770 msgr2=0x55b54aaa5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa83770 0x55b54aaa5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9553400640 1 -- 172.21.15.94:0/3950052580 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54aa884b0 con 0x55b54aa82200 2024-06-03T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.828+0000 7f9552a00640 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa83770 0x55b54aaa5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.829+0000 7f9553400640 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa84470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9544009860 tx=0x7f954403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c5bdc28943b494 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.829+0000 7f953fe00640 1 -- 172.21.15.94:0/3950052580 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544037040 con 0x55b54aa82200 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.829+0000 7f953fe00640 1 -- 172.21.15.94:0/3950052580 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9544003b90 con 0x55b54aa82200 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.829+0000 7f953fe00640 1 -- 172.21.15.94:0/3950052580 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954404bc90 con 0x55b54aa82200 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 -- 172.21.15.94:0/3950052580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 msgr2=0x55b54aa84470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa84470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9544009860 tx=0x7f954403d000 comp rx=0 tx=0).stop 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 -- 172.21.15.94:0/3950052580 shutdown_connections 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa8cf80 0x55b54aaa58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa83770 0x55b54aaa5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 --2- 172.21.15.94:0/3950052580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa82200 0x55b54aa84470 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 -- 172.21.15.94:0/3950052580 >> 172.21.15.94:0/3950052580 conn(0x55b54aa4af80 msgr2=0x55b54aa83d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 -- 172.21.15.94:0/3950052580 shutdown_connections 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 -- 172.21.15.94:0/3950052580 wait complete. 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.830+0000 7f955d42a740 1 -- start start 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 1 Processor -- start 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa90ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa91410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa8cf80 0x55b54aa91950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f95541b67d0 con 0x55b54aa8cf80 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95541b6070 con 0x55b54aa82200 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f95541b3270 con 0x55b54aa83770 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f955d42a740 10 client.0 _ll_register_callbacks cb 0x7f954c097270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f9553e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa8cf80 0x55b54aa91950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f9553e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa8cf80 0x55b54aa91950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60928/0 (socket says 172.21.15.94:60928) 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f9553e00640 1 -- 172.21.15.94:0/282192884 learned_addr learned my addr 172.21.15.94:0/282192884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f9553400640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa90ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.831+0000 7f9552a00640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa91410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f9553e00640 1 -- 172.21.15.94:0/282192884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 msgr2=0x55b54aa91410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f9553e00640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa91410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f9553e00640 1 -- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 msgr2=0x55b54aa90ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f9553e00640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa90ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f9553e00640 1 -- 172.21.15.94:0/282192884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54ab07330 con 0x55b54aa8cf80 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f9553400640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa90ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f9553e00640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa8cf80 0x55b54aa91950 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f954c09b680 tx=0x7f954c09beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccef6431bd8cdc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f953d600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f953d600640 1 -- 172.21.15.94:0/282192884 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c097990 con 0x55b54aa8cf80 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f953d600640 1 -- 172.21.15.94:0/282192884 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9475+0+0 (secure 0 0 0) 0x7f954c00bb90 con 0x55b54aa8cf80 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9544003b90 con 0x55b54aa8cf80 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f953d600640 1 -- 172.21.15.94:0/282192884 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f954c09b050 con 0x55b54aa8cf80 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f955d42a740 10 client.15876 fetch_fsmap learned FSMap version 43 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.832+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b54ab085a0 con 0x55b54aa8cf80 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f953d600640 1 -- 172.21.15.94:0/282192884 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9544003b90 con 0x55b54aa8cf80 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f953d600640 1 -- 172.21.15.94:0/282192884 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f954c09ab40 con 0x55b54aa8cf80 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f955d42a740 10 client.15876 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f955d42a740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f955d42a740 20 client.15876 populate_metadata read hostname 'smithi094' 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f955d42a740 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54ab08bc0 0x55b54ab0afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f955d42a740 4 client.15876 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b54a9edea0 con 0x55b54ab08bc0 2024-06-03T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.833+0000 7f9553400640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54ab08bc0 0x55b54ab0afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.834+0000 7f953d600640 10 client.15876 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.834+0000 7f9553400640 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54ab08bc0 0x55b54ab0afe0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85275ca1a1cb494 server_cookie=b2e4e8edd95fe37d in_seq=0 out_seq=0 2024-06-03T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f953d600640 1 -- 172.21.15.94:0/282192884 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b54a9edea0 con 0x55b54ab08bc0 2024-06-03T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f953d600640 10 client.15876 handle_command_reply: tid=0 2024-06-03T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 client.15876 shutdown 2024-06-03T01:18:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa8cf80 msgr2=0x55b54aa91950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa8cf80 0x55b54aa91950 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f954c09b680 tx=0x7f954c09beb0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 shutdown_connections 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b54aa8cf80 0x55b54aa91950 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b54aa83770 0x55b54aa91410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55b54ab08bc0 0x55b54ab0afe0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 --2- 172.21.15.94:0/282192884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b54aa82200 0x55b54aa90ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 >> 172.21.15.94:0/282192884 conn(0x55b54aa4af80 msgr2=0x55b54aa8a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f953d600640 0 client.15876 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.856+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 shutdown_connections 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.857+0000 7f955d42a740 1 -- 172.21.15.94:0/282192884 wait complete. 2024-06-03T01:18:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.857+0000 7f955d42a740 20 client.15876 trim_cache size 0 max 16384 2024-06-03T01:18:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:18:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:18:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95380a2d70 msgr2=0x7f95380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95380a2d70 0x7f95380a5230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9544000c00 tx=0x7f95440056d0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f955410de70 msgr2=0x7f95541a2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f955410de70 0x7f95541a2f90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f954c00efc0 tx=0x7f954c00c800 comp rx=0 tx=0).stop 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 shutdown_connections 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95380a2d70 0x7f95380a5230 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f955410de70 0x7f95541a2f90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9554108860 0x7f955419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 --2- 172.21.15.94:0/951462890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9554107ee0 0x7f955419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:18:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 >> 172.21.15.94:0/951462890 conn(0x7f9554077780 msgr2=0x7f9554078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:18:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 shutdown_connections 2024-06-03T01:18:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:18:59.858+0000 7f9559000640 1 -- 172.21.15.94:0/951462890 wait complete. 2024-06-03T01:19:00.111 INFO:tasks.workunit.client.0.smithi094.stdout: 1 195775 2.12 MB/sec execute 307 sec latency 2609.759 ms 2024-06-03T01:19:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:00 smithi107 ceph-mon[36622]: pgmap v266: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 728 KiB/s wr, 65 op/s 2024-06-03T01:19:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:00 smithi094 ceph-mon[32119]: pgmap v266: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 728 KiB/s wr, 65 op/s 2024-06-03T01:19:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:00 smithi179 ceph-mon[37041]: pgmap v266: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 728 KiB/s wr, 65 op/s 2024-06-03T01:19:01.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 196126 2.12 MB/sec execute 308 sec latency 94.557 ms 2024-06-03T01:19:02.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 196685 2.13 MB/sec execute 309 sec latency 93.813 ms 2024-06-03T01:19:02.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:02 smithi107 ceph-mon[36622]: pgmap v267: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 729 KiB/s wr, 65 op/s 2024-06-03T01:19:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:02 smithi094 ceph-mon[32119]: pgmap v267: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 729 KiB/s wr, 65 op/s 2024-06-03T01:19:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:02 smithi179 ceph-mon[37041]: pgmap v267: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 729 KiB/s wr, 65 op/s 2024-06-03T01:19:03.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 196788 2.12 MB/sec execute 310 sec latency 893.768 ms 2024-06-03T01:19:04.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 198123 2.14 MB/sec execute 311 sec latency 905.701 ms 2024-06-03T01:19:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:04 smithi107 ceph-mon[36622]: pgmap v268: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-06-03T01:19:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:04 smithi094 ceph-mon[32119]: pgmap v268: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-06-03T01:19:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:04 smithi179 ceph-mon[37041]: pgmap v268: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-06-03T01:19:05.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 199058 2.15 MB/sec execute 312 sec latency 31.459 ms 2024-06-03T01:19:05.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:05 smithi107 ceph-mon[36622]: osdmap e106: 12 total, 12 up, 12 in 2024-06-03T01:19:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:05 smithi094 ceph-mon[32119]: osdmap e106: 12 total, 12 up, 12 in 2024-06-03T01:19:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:05 smithi179 ceph-mon[37041]: osdmap e106: 12 total, 12 up, 12 in 2024-06-03T01:19:06.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 199419 2.14 MB/sec execute 313 sec latency 491.733 ms 2024-06-03T01:19:06.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:06 smithi107 ceph-mon[36622]: pgmap v270: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 821 KiB/s wr, 123 op/s 2024-06-03T01:19:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:06 smithi094 ceph-mon[32119]: pgmap v270: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 821 KiB/s wr, 123 op/s 2024-06-03T01:19:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:06 smithi179 ceph-mon[37041]: pgmap v270: 129 pgs: 129 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 821 KiB/s wr, 123 op/s 2024-06-03T01:19:07.112 INFO:tasks.workunit.client.0.smithi094.stdout: 1 200331 2.15 MB/sec execute 314 sec latency 117.505 ms 2024-06-03T01:19:08.113 INFO:tasks.workunit.client.0.smithi094.stdout: 1 200331 2.15 MB/sec execute 315 sec latency 1117.668 ms 2024-06-03T01:19:08.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:08 smithi107 ceph-mon[36622]: pgmap v271: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 832 KiB/s wr, 151 op/s 2024-06-03T01:19:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:08 smithi094 ceph-mon[32119]: pgmap v271: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 832 KiB/s wr, 151 op/s 2024-06-03T01:19:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:08 smithi179 ceph-mon[37041]: pgmap v271: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 832 KiB/s wr, 151 op/s 2024-06-03T01:19:09.113 INFO:tasks.workunit.client.0.smithi094.stdout: 1 200331 2.14 MB/sec execute 316 sec latency 2117.813 ms 2024-06-03T01:19:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:09 smithi107 ceph-mon[36622]: osdmap e107: 12 total, 12 up, 12 in 2024-06-03T01:19:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:09 smithi107 ceph-mon[36622]: pgmap v273: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-06-03T01:19:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:09 smithi094 ceph-mon[32119]: osdmap e107: 12 total, 12 up, 12 in 2024-06-03T01:19:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:09 smithi094 ceph-mon[32119]: pgmap v273: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-06-03T01:19:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:09 smithi179 ceph-mon[37041]: osdmap e107: 12 total, 12 up, 12 in 2024-06-03T01:19:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:09 smithi179 ceph-mon[37041]: pgmap v273: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-06-03T01:19:10.113 INFO:tasks.workunit.client.0.smithi094.stdout: 1 200331 2.13 MB/sec execute 317 sec latency 3117.951 ms 2024-06-03T01:19:11.113 INFO:tasks.workunit.client.0.smithi094.stdout: 1 201107 2.13 MB/sec execute 318 sec latency 3323.260 ms 2024-06-03T01:19:12.113 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202424 2.16 MB/sec execute 319 sec latency 19.709 ms 2024-06-03T01:19:12.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:12 smithi107 ceph-mon[36622]: pgmap v274: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 584 KiB/s wr, 86 op/s 2024-06-03T01:19:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:12 smithi094 ceph-mon[32119]: pgmap v274: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 584 KiB/s wr, 86 op/s 2024-06-03T01:19:12.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:12 smithi179 ceph-mon[37041]: pgmap v274: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 584 KiB/s wr, 86 op/s 2024-06-03T01:19:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.998+0000 7ff391800640 1 -- 172.21.15.94:0/3045286758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c103e10 msgr2=0x7ff38c1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.998+0000 7ff391800640 1 --2- 172.21.15.94:0/3045286758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c103e10 0x7ff38c1041f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009900 tx=0x7ff37c02a3f0 comp rx=0 tx=0).stop 2024-06-03T01:19:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.999+0000 7ff391800640 1 -- 172.21.15.94:0/3045286758 shutdown_connections 2024-06-03T01:19:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.999+0000 7ff391800640 1 --2- 172.21.15.94:0/3045286758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c108ee0 0x7ff38c110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.999+0000 7ff391800640 1 --2- 172.21.15.94:0/3045286758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff38c1047c0 0x7ff38c108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.999+0000 7ff391800640 1 --2- 172.21.15.94:0/3045286758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c103e10 0x7ff38c1041f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.999+0000 7ff391800640 1 -- 172.21.15.94:0/3045286758 >> 172.21.15.94:0/3045286758 conn(0x7ff38c0ff6d0 msgr2=0x7ff38c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.999+0000 7ff391800640 1 -- 172.21.15.94:0/3045286758 shutdown_connections 2024-06-03T01:19:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:12.999+0000 7ff391800640 1 -- 172.21.15.94:0/3045286758 wait complete. 2024-06-03T01:19:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 Processor -- start 2024-06-03T01:19:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 -- start start 2024-06-03T01:19:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c103e10 0x7ff38c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff38c108ee0 0x7ff38c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c19ed20 0x7ff38c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c113fe0 con 0x7ff38c103e10 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c113e60 con 0x7ff38c19ed20 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.000+0000 7ff391800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c114160 con 0x7ff38c108ee0 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c103e10 0x7ff38c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c19ed20 0x7ff38c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c103e10 0x7ff38c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54238/0 (socket says 172.21.15.94:54238) 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38aa00640 1 -- 172.21.15.94:0/1954243635 learned_addr learned my addr 172.21.15.94:0/1954243635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff382000640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff38c108ee0 0x7ff38c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38b400640 1 -- 172.21.15.94:0/1954243635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff38c108ee0 msgr2=0x7ff38c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38b400640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff38c108ee0 0x7ff38c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38b400640 1 -- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c103e10 msgr2=0x7ff38c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38b400640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c103e10 0x7ff38c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38b400640 1 -- 172.21.15.94:0/1954243635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c19f810 con 0x7ff38c19ed20 2024-06-03T01:19:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.001+0000 7ff38aa00640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c103e10 0x7ff38c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.002+0000 7ff38b400640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c19ed20 0x7ff38c19f1d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fd0 tx=0x7ff38400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.002+0000 7ff383e00640 1 -- 172.21.15.94:0/1954243635 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38400ce20 con 0x7ff38c19ed20 2024-06-03T01:19:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.002+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c19fb00 con 0x7ff38c19ed20 2024-06-03T01:19:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.002+0000 7ff383e00640 1 -- 172.21.15.94:0/1954243635 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384010430 con 0x7ff38c19ed20 2024-06-03T01:19:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.002+0000 7ff383e00640 1 -- 172.21.15.94:0/1954243635 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38400f600 con 0x7ff38c19ed20 2024-06-03T01:19:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.002+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c1a72c0 con 0x7ff38c19ed20 2024-06-03T01:19:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.005+0000 7ff383e00640 1 -- 172.21.15.94:0/1954243635 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff3840105e0 con 0x7ff38c19ed20 2024-06-03T01:19:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.005+0000 7ff3947ad740 1 Processor -- start 2024-06-03T01:19:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff3947ad740 1 -- start start 2024-06-03T01:19:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb52ffcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb52ffd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb5300a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff3947ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bb52f61420 con 0x55bb52ffd040 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff3947ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bb52f08c50 con 0x55bb52ffe5b0 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff3947ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bb52f5d3e0 con 0x55bb53007dc0 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb52ffd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb52ffd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54242/0 (socket says 172.21.15.94:54242) 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff38aa00640 1 -- 172.21.15.94:0/3256548509 learned_addr learned my addr 172.21.15.94:0/3256548509 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.006+0000 7ff382000640 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb52ffcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff382000640 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb5300a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff38aa00640 1 -- 172.21.15.94:0/3256548509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 msgr2=0x55bb5300a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff38aa00640 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb5300a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff38aa00640 1 -- 172.21.15.94:0/3256548509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 msgr2=0x55bb52ffcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff38aa00640 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb52ffcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff38aa00640 1 -- 172.21.15.94:0/3256548509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb52c75210 con 0x55bb52ffd040 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff382000640 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb52ffcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff38aa00640 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb52ffd4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005c40 tx=0x7ff37c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9178cb5fdcff4a83 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff383e00640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff37be00640 1 -- 172.21.15.94:0/3256548509 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c040070 con 0x55bb52ffd040 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff37be00640 1 -- 172.21.15.94:0/3256548509 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37c0047a0 con 0x55bb52ffd040 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.007+0000 7ff37be00640 1 -- 172.21.15.94:0/3256548509 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c035040 con 0x55bb52ffd040 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff383e00640 1 -- 172.21.15.94:0/1954243635 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7ff3840cfa70 con 0x7ff38c19ed20 2024-06-03T01:19:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff382000640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 -- 172.21.15.94:0/3256548509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 msgr2=0x55bb52ffd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb52ffd4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005c40 tx=0x7ff37c005dd0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 -- 172.21.15.94:0/3256548509 shutdown_connections 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb5300a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb52ffd4c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3256548509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb52ffcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 -- 172.21.15.94:0/3256548509 >> 172.21.15.94:0/3256548509 conn(0x55bb52fc5f80 msgr2=0x55bb52ff9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 -- 172.21.15.94:0/3256548509 shutdown_connections 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff382000640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55bb52ffa7c0 tx=0x7ff3740096a0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.008+0000 7ff3947ad740 1 -- 172.21.15.94:0/3256548509 wait complete. 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.009+0000 7ff3947ad740 1 -- start start 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.009+0000 7ff3947ad740 1 Processor -- start 2024-06-03T01:19:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffd040 0x55bb53004d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb530057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff3947ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55bb5300ab30 con 0x55bb53007dc0 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff3947ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bb52f1c0b0 con 0x55bb52ffe5b0 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff3947ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bb5300acb0 con 0x55bb52ffd040 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff3947ad740 10 client.0 _ll_register_callbacks cb 0x55bb52f07f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb530057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb530057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54258/0 (socket says 172.21.15.94:54258) 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38b400640 1 -- 172.21.15.94:0/2260484734 learned_addr learned my addr 172.21.15.94:0/2260484734 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38aa00640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffd040 0x55bb53004d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff382000640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38aa00640 1 -- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 msgr2=0x55bb530052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38aa00640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38aa00640 1 -- 172.21.15.94:0/2260484734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 msgr2=0x55bb530057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38aa00640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb530057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38aa00640 1 -- 172.21.15.94:0/2260484734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb52d41620 con 0x55bb52ffd040 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.010+0000 7ff38b400640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb530057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff38aa00640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffd040 0x55bb53004d60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff37c02a9e0 tx=0x7ff37c02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c0d78400093100 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff378c00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff378c00640 1 -- 172.21.15.94:0/2260484734 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c04c020 con 0x55bb52ffd040 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37c0047a0 con 0x55bb52ffd040 2024-06-03T01:19:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff378c00640 1 -- 172.21.15.94:0/2260484734 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7ff37c0053a0 con 0x55bb52ffd040 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff378c00640 1 -- 172.21.15.94:0/2260484734 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff37c057050 con 0x55bb52ffd040 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff3947ad740 10 client.26017 fetch_fsmap learned FSMap version 43 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.011+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bb5300ab30 con 0x55bb52ffd040 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff378c00640 1 -- 172.21.15.94:0/2260484734 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37c0047a0 con 0x55bb52ffd040 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff378c00640 1 -- 172.21.15.94:0/2260484734 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7ff37c0315d0 con 0x55bb52ffd040 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff3947ad740 10 client.26017 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff3947ad740 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff3947ad740 20 client.26017 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff3947ad740 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53018010 0x55bb5301a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff3947ad740 4 client.26017 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:19:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb52f6eb50 con 0x55bb53018010 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.012+0000 7ff382000640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53018010 0x55bb5301a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.013+0000 7ff382000640 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53018010 0x55bb5301a420 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ad303f3bc1cb3c server_cookie=f86460b7aa85ee1d in_seq=0 out_seq=0 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.013+0000 7ff378c00640 10 client.26017 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.013+0000 7ff378c00640 1 -- 172.21.15.94:0/2260484734 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb52f6eb50 con 0x55bb53018010 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.013+0000 7ff378c00640 10 client.26017 handle_command_reply: tid=0 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.013+0000 7ff3947ad740 1 client.26017 shutdown 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffd040 msgr2=0x55bb53004d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffd040 0x55bb53004d60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff37c02a9e0 tx=0x7ff37c02a3f0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 shutdown_connections 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb530057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53018010 0x55bb5301a420 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 --2- 172.21.15.94:0/2260484734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffd040 0x55bb53004d60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff378c00640 0 client.26017 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 >> 172.21.15.94:0/2260484734 conn(0x55bb52fc5f80 msgr2=0x55bb52fff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 shutdown_connections 2024-06-03T01:19:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 1 -- 172.21.15.94:0/2260484734 wait complete. 2024-06-03T01:19:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.014+0000 7ff3947ad740 20 client.26017 trim_cache size 0 max 16384 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 Processor -- start 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 -- start start 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffd040 0x55bb53003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffe5b0 0x55bb52fff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb52ff8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1b0fe0 con 0x55bb53007dc0 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1b2cd0 con 0x55bb52ffd040 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff3947ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1b1710 con 0x55bb52ffe5b0 2024-06-03T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffd040 0x55bb53003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.031+0000 7ff38b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb52ff8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffd040 0x55bb53003ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:39358/0 (socket says 172.21.15.94:39358) 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 -- 172.21.15.94:0/1039828797 learned_addr learned my addr 172.21.15.94:0/1039828797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff382000640 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffe5b0 0x55bb52fff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 -- 172.21.15.94:0/1039828797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffe5b0 msgr2=0x55bb52fff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffe5b0 0x55bb52fff2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 -- 172.21.15.94:0/1039828797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 msgr2=0x55bb52ff8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb52ff8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 -- 172.21.15.94:0/1039828797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb52ffdb10 con 0x55bb52ffd040 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff382000640 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffe5b0 0x55bb52fff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:19:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff38aa00640 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffd040 0x55bb53003ec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff37c031460 tx=0x7ff37c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5388547ce8a5bf6a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff37b400640 1 -- 172.21.15.94:0/1039828797 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c04c070 con 0x55bb52ffd040 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff37b400640 1 -- 172.21.15.94:0/1039828797 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37c004770 con 0x55bb52ffd040 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.032+0000 7ff37b400640 1 -- 172.21.15.94:0/1039828797 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c040070 con 0x55bb52ffd040 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 -- 172.21.15.94:0/1039828797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffd040 msgr2=0x55bb53003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffd040 0x55bb53003ec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff37c031460 tx=0x7ff37c009260 comp rx=0 tx=0).stop 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 -- 172.21.15.94:0/1039828797 shutdown_connections 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb53007dc0 0x55bb52ff8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb52ffe5b0 0x55bb52fff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 --2- 172.21.15.94:0/1039828797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffd040 0x55bb53003ec0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 -- 172.21.15.94:0/1039828797 >> 172.21.15.94:0/1039828797 conn(0x55bb52fc5f80 msgr2=0x55bb52ffeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.033+0000 7ff3947ad740 1 -- 172.21.15.94:0/1039828797 shutdown_connections 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.034+0000 7ff3947ad740 1 -- 172.21.15.94:0/1039828797 wait complete. 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.034+0000 7ff3947ad740 1 -- start start 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.034+0000 7ff3947ad740 1 Processor -- start 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb53010190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff3947ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb53002020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff3947ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1b2540 con 0x55bb52ffd040 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff3947ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1b1de0 con 0x55bb52ffe5b0 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff3947ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c1aefe0 con 0x55bb53007dc0 2024-06-03T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff3947ad740 10 client.0 _ll_register_callbacks cb 0x7ff37c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb53010190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff382000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb53002020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff382000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:39362/0 (socket says 172.21.15.94:39362) 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb53002020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:35592/0 (socket says 172.21.15.94:35592) 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb53010190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54282/0 (socket says 172.21.15.94:54282) 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff382000640 1 -- 172.21.15.94:0/3021149123 learned_addr learned my addr 172.21.15.94:0/3021149123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38b400640 1 -- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 msgr2=0x55bb530106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38b400640 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38b400640 1 -- 172.21.15.94:0/3021149123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 msgr2=0x55bb53010190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38b400640 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb53010190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff38b400640 1 -- 172.21.15.94:0/3021149123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb53002850 con 0x55bb53007dc0 2024-06-03T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.035+0000 7ff382000640 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.036+0000 7ff38b400640 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb53002020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff384005fd0 tx=0x7ff38409c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fea8023d95d3173 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.036+0000 7ff378c00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.036+0000 7ff378c00640 1 -- 172.21.15.94:0/3021149123 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840b3070 con 0x55bb53007dc0 2024-06-03T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.036+0000 7ff378c00640 1 -- 172.21.15.94:0/3021149123 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7ff3840ba500 con 0x55bb53007dc0 2024-06-03T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.036+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37c004770 con 0x55bb53007dc0 2024-06-03T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.036+0000 7ff378c00640 1 -- 172.21.15.94:0/3021149123 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff384082050 con 0x55bb53007dc0 2024-06-03T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 10 client.26023 fetch_fsmap learned FSMap version 43 2024-06-03T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bb5301fec0 con 0x55bb53007dc0 2024-06-03T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff378c00640 1 -- 172.21.15.94:0/3021149123 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37c004770 con 0x55bb53007dc0 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff378c00640 1 -- 172.21.15.94:0/3021149123 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7ff38401f020 con 0x55bb53007dc0 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 10 client.26023 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 10 client.26023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 20 client.26023 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53083150 0x55bb53085570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 4 client.26023 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb52f68ea0 con 0x55bb53083150 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.037+0000 7ff38aa00640 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53083150 0x55bb53085570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.038+0000 7ff38aa00640 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53083150 0x55bb53085570 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431a8a457a6ed405 server_cookie=f13b8a1832e3fc84 in_seq=0 out_seq=0 2024-06-03T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.038+0000 7ff378c00640 10 client.26023 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff378c00640 1 -- 172.21.15.94:0/3021149123 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bb52f68ea0 con 0x55bb53083150 2024-06-03T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff378c00640 10 client.26023 handle_command_reply: tid=0 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 client.26023 shutdown 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 msgr2=0x55bb53002020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb53002020 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff384005fd0 tx=0x7ff38409c3d0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 shutdown_connections 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb53007dc0 0x55bb53002020 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55bb53083150 0x55bb53085570 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb52ffe5b0 0x55bb530106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 --2- 172.21.15.94:0/3021149123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bb52ffd040 0x55bb53010190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff378c00640 0 client.26023 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.062+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 >> 172.21.15.94:0/3021149123 conn(0x55bb52fc5f80 msgr2=0x55bb53005590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.063+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 shutdown_connections 2024-06-03T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.063+0000 7ff3947ad740 1 -- 172.21.15.94:0/3021149123 wait complete. 2024-06-03T01:19:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.063+0000 7ff3947ad740 20 client.26023 trim_cache size 0 max 16384 2024-06-03T01:19:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:19:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:19:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.063+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff36c0a2e20 msgr2=0x7ff36c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.063+0000 7ff391800640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55bb52ffa7c0 tx=0x7ff3740096a0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.063+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c19ed20 msgr2=0x7ff38c19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.063+0000 7ff391800640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c19ed20 0x7ff38c19f1d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fd0 tx=0x7ff38400ef10 comp rx=0 tx=0).stop 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 shutdown_connections 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff36c0a2e20 0x7ff36c0a52e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff38c19ed20 0x7ff38c19f1d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff38c108ee0 0x7ff38c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 --2- 172.21.15.94:0/1954243635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff38c103e10 0x7ff38c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:13.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 >> 172.21.15.94:0/1954243635 conn(0x7ff38c0ff6d0 msgr2=0x7ff38c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 shutdown_connections 2024-06-03T01:19:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:13.064+0000 7ff391800640 1 -- 172.21.15.94:0/1954243635 wait complete. 2024-06-03T01:19:13.113 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202908 2.16 MB/sec execute 320 sec latency 239.905 ms 2024-06-03T01:19:14.113 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202908 2.15 MB/sec execute 321 sec latency 1240.075 ms 2024-06-03T01:19:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:14 smithi094 ceph-mon[32119]: pgmap v275: 129 pgs: 129 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 996 KiB/s wr, 123 op/s 2024-06-03T01:19:14.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:14 smithi179 ceph-mon[37041]: pgmap v275: 129 pgs: 129 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 996 KiB/s wr, 123 op/s 2024-06-03T01:19:14.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:14 smithi107 ceph-mon[36622]: pgmap v275: 129 pgs: 129 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 996 KiB/s wr, 123 op/s 2024-06-03T01:19:15.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202908 2.14 MB/sec execute 322 sec latency 2240.256 ms 2024-06-03T01:19:16.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202968 2.14 MB/sec execute 323 sec latency 3180.969 ms 2024-06-03T01:19:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:16 smithi094 ceph-mon[32119]: pgmap v276: 129 pgs: 129 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 875 KiB/s wr, 108 op/s 2024-06-03T01:19:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:16 smithi179 ceph-mon[37041]: pgmap v276: 129 pgs: 129 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 875 KiB/s wr, 108 op/s 2024-06-03T01:19:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:16 smithi107 ceph-mon[36622]: pgmap v276: 129 pgs: 129 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 875 KiB/s wr, 108 op/s 2024-06-03T01:19:17.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202976 2.13 MB/sec execute 324 sec latency 984.046 ms 2024-06-03T01:19:18.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202976 2.12 MB/sec execute 325 sec latency 1984.150 ms 2024-06-03T01:19:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:18 smithi094 ceph-mon[32119]: pgmap v277: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 721 KiB/s wr, 78 op/s 2024-06-03T01:19:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:18 smithi179 ceph-mon[37041]: pgmap v277: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 721 KiB/s wr, 78 op/s 2024-06-03T01:19:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:18 smithi107 ceph-mon[36622]: pgmap v277: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 721 KiB/s wr, 78 op/s 2024-06-03T01:19:19.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202976 2.12 MB/sec execute 326 sec latency 2984.301 ms 2024-06-03T01:19:20.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 202976 2.11 MB/sec execute 327 sec latency 3984.393 ms 2024-06-03T01:19:20.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:20 smithi094 ceph-mon[32119]: pgmap v278: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 697 KiB/s wr, 75 op/s 2024-06-03T01:19:20.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:20 smithi179 ceph-mon[37041]: pgmap v278: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 697 KiB/s wr, 75 op/s 2024-06-03T01:19:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:20 smithi107 ceph-mon[36622]: pgmap v278: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 697 KiB/s wr, 75 op/s 2024-06-03T01:19:21.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 203053 2.11 MB/sec execute 328 sec latency 4921.107 ms 2024-06-03T01:19:22.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 203878 2.11 MB/sec execute 329 sec latency 223.653 ms 2024-06-03T01:19:22.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:22 smithi094 ceph-mon[32119]: pgmap v279: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 601 KiB/s wr, 65 op/s 2024-06-03T01:19:22.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:22 smithi179 ceph-mon[37041]: pgmap v279: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 601 KiB/s wr, 65 op/s 2024-06-03T01:19:22.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:22 smithi107 ceph-mon[36622]: pgmap v279: 129 pgs: 129 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 601 KiB/s wr, 65 op/s 2024-06-03T01:19:23.114 INFO:tasks.workunit.client.0.smithi094.stdout: 1 203878 2.11 MB/sec execute 330 sec latency 1223.756 ms 2024-06-03T01:19:24.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 203878 2.10 MB/sec execute 331 sec latency 2223.935 ms 2024-06-03T01:19:24.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:24 smithi094 ceph-mon[32119]: pgmap v280: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 KiB/s wr, 86 op/s 2024-06-03T01:19:24.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:24 smithi179 ceph-mon[37041]: pgmap v280: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 KiB/s wr, 86 op/s 2024-06-03T01:19:24.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:24 smithi107 ceph-mon[36622]: pgmap v280: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 711 KiB/s wr, 86 op/s 2024-06-03T01:19:25.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 203878 2.09 MB/sec execute 332 sec latency 3224.035 ms 2024-06-03T01:19:26.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 205088 2.11 MB/sec execute 333 sec latency 3425.522 ms 2024-06-03T01:19:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 -- 172.21.15.94:0/357910644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89100746f0 msgr2=0x7f891010daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 --2- 172.21.15.94:0/357910644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89100746f0 0x7f891010daa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009860 tx=0x7f88f802a3d0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 -- 172.21.15.94:0/357910644 shutdown_connections 2024-06-03T01:19:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 --2- 172.21.15.94:0/357910644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f891010dfe0 0x7f8910110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 --2- 172.21.15.94:0/357910644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89100746f0 0x7f891010daa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 --2- 172.21.15.94:0/357910644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8910073dd0 0x7f89100741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 -- 172.21.15.94:0/357910644 >> 172.21.15.94:0/357910644 conn(0x7f89100fd2f0 msgr2=0x7f89100ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 -- 172.21.15.94:0/357910644 shutdown_connections 2024-06-03T01:19:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.209+0000 7f890fe00640 1 -- 172.21.15.94:0/357910644 wait complete. 2024-06-03T01:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.210+0000 7f890fe00640 1 Processor -- start 2024-06-03T01:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.210+0000 7f890fe00640 1 -- start start 2024-06-03T01:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8910073dd0 0x7f891019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89100746f0 0x7f891019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f891010dfe0 0x7f891019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8910113d80 con 0x7f89100746f0 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8910113c00 con 0x7f8910073dd0 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8910113f00 con 0x7f891010dfe0 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8910073dd0 0x7f891019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8910073dd0 0x7f891019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51818/0 (socket says 172.21.15.94:51818) 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 -- 172.21.15.94:0/1500286257 learned_addr learned my addr 172.21.15.94:0/1500286257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890e000640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89100746f0 0x7f891019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890f400640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f891010dfe0 0x7f891019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 -- 172.21.15.94:0/1500286257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f891010dfe0 msgr2=0x7f891019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f891010dfe0 0x7f891019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 -- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89100746f0 msgr2=0x7f891019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89100746f0 0x7f891019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.211+0000 7f890ea00640 1 -- 172.21.15.94:0/1500286257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891019f1f0 con 0x7f8910073dd0 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.212+0000 7f890e000640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89100746f0 0x7f891019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.212+0000 7f890ea00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8910073dd0 0x7f891019a200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f890000c420 tx=0x7f8900009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.212+0000 7f8907400640 1 -- 172.21.15.94:0/1500286257 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900003970 con 0x7f8910073dd0 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.212+0000 7f8907400640 1 -- 172.21.15.94:0/1500286257 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8900004020 con 0x7f8910073dd0 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.212+0000 7f8907400640 1 -- 172.21.15.94:0/1500286257 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890000bad0 con 0x7f8910073dd0 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.212+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891019f4e0 con 0x7f8910073dd0 2024-06-03T01:19:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.212+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101a6d70 con 0x7f8910073dd0 2024-06-03T01:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.216+0000 7f8907400640 1 -- 172.21.15.94:0/1500286257 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f890001d020 con 0x7f8910073dd0 2024-06-03T01:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.216+0000 7f8918605740 1 Processor -- start 2024-06-03T01:19:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.216+0000 7f8918605740 1 -- start start 2024-06-03T01:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.216+0000 7f8918605740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463449770 0x55d463447cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.216+0000 7f8918605740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d463448680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.216+0000 7f8918605740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d463455370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f8918605740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d4633ac420 con 0x55d463449770 2024-06-03T01:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f8918605740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d463353c50 con 0x55d463448200 2024-06-03T01:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f8918605740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d4633a83e0 con 0x55d463452f80 2024-06-03T01:19:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f890e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d463455370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f890ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d463448680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f890e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d463455370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:58600/0 (socket says 172.21.15.94:58600) 2024-06-03T01:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f890e000640 1 -- 172.21.15.94:0/942576795 learned_addr learned my addr 172.21.15.94:0/942576795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.217+0000 7f890f400640 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463449770 0x55d463447cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890e000640 1 -- 172.21.15.94:0/942576795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 msgr2=0x55d463448680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890e000640 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d463448680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890e000640 1 -- 172.21.15.94:0/942576795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463449770 msgr2=0x55d463447cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890e000640 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463449770 0x55d463447cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890e000640 1 -- 172.21.15.94:0/942576795 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4630c0210 con 0x55d463452f80 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890ea00640 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d463448680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890e000640 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d463455370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88f8005de0 tx=0x7f88f8030ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e27cd373d84cf457 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f8907400640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f88f40a2cf0 0x7f88f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f88ffe00640 1 -- 172.21.15.94:0/942576795 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8004050 con 0x55d463452f80 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f88ffe00640 1 -- 172.21.15.94:0/942576795 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f8030880 con 0x55d463452f80 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f8907400640 1 -- 172.21.15.94:0/1500286257 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7f89000cf830 con 0x7f8910073dd0 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f88ffe00640 1 -- 172.21.15.94:0/942576795 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f80046c0 con 0x55d463452f80 2024-06-03T01:19:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.218+0000 7f890f400640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f88f40a2cf0 0x7f88f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 -- 172.21.15.94:0/942576795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 msgr2=0x55d463455370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d463455370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88f8005de0 tx=0x7f88f8030ab0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f890f400640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f88f40a2cf0 0x7f88f40a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8908005fd0 tx=0x7f8908005e40 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 -- 172.21.15.94:0/942576795 shutdown_connections 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d463455370 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d463448680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 --2- 172.21.15.94:0/942576795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463449770 0x55d463447cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 -- 172.21.15.94:0/942576795 >> 172.21.15.94:0/942576795 conn(0x55d463410f80 msgr2=0x55d4634447f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.219+0000 7f8918605740 1 -- 172.21.15.94:0/942576795 shutdown_connections 2024-06-03T01:19:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.220+0000 7f8918605740 1 -- 172.21.15.94:0/942576795 wait complete. 2024-06-03T01:19:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.220+0000 7f8918605740 1 -- start start 2024-06-03T01:19:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.220+0000 7f8918605740 1 Processor -- start 2024-06-03T01:19:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f8918605740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d46344ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f8918605740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d4634504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f8918605740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463452f80 0x55d463450a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f8918605740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d463455cf0 con 0x55d463452f80 2024-06-03T01:19:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f8918605740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d4633670b0 con 0x55d463448200 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f8918605740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d463455e70 con 0x55d463449770 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f8918605740 10 client.0 _ll_register_callbacks cb 0x55d463352f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d4634504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d46344ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463452f80 0x55d463450a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d4634504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:58612/0 (socket says 172.21.15.94:58612) 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d46344ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51832/0 (socket says 172.21.15.94:51832) 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890e000640 1 -- 172.21.15.94:0/1985054724 learned_addr learned my addr 172.21.15.94:0/1985054724 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890ea00640 1 -- 172.21.15.94:0/1985054724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 msgr2=0x55d4634504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890ea00640 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d4634504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890ea00640 1 -- 172.21.15.94:0/1985054724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463452f80 msgr2=0x55d463450a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890ea00640 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463452f80 0x55d463450a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.221+0000 7f890ea00640 1 -- 172.21.15.94:0/1985054724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46318c620 con 0x55d463448200 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f890ea00640 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d46344ffa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f89000ccbc0 tx=0x7f8900097c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef184c048c2b8ca server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f88fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f88fcc00640 1 -- 172.21.15.94:0/1985054724 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890009b2f0 con 0x55d463448200 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f88fcc00640 1 -- 172.21.15.94:0/1985054724 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7f89000cf440 con 0x55d463448200 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f8030880 con 0x55d463448200 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f88fcc00640 1 -- 172.21.15.94:0/1985054724 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f89000041d0 con 0x55d463448200 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f8918605740 10 client.26018 fetch_fsmap learned FSMap version 43 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.222+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d463462060 con 0x55d463448200 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f88fcc00640 1 -- 172.21.15.94:0/1985054724 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88f8030880 con 0x55d463448200 2024-06-03T01:19:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f88fcc00640 1 -- 172.21.15.94:0/1985054724 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f890000b040 con 0x55d463448200 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f8918605740 10 client.26018 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f8918605740 10 client.26018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f8918605740 20 client.26018 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f8918605740 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d463463570 0x55d4634659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f8918605740 4 client.26018 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4633b9b50 con 0x55d463463570 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.223+0000 7f890e000640 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d463463570 0x55d4634659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.224+0000 7f890e000640 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d463463570 0x55d4634659d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46473622c14a9c28 server_cookie=3da13577cef69f34 in_seq=0 out_seq=0 2024-06-03T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.224+0000 7f88fcc00640 10 client.26018 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f88fcc00640 1 -- 172.21.15.94:0/1985054724 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d4633b9b50 con 0x55d463463570 2024-06-03T01:19:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f88fcc00640 10 client.26018 handle_command_reply: tid=0 2024-06-03T01:19:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 client.26018 shutdown 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 msgr2=0x55d46344ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d46344ffa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f89000ccbc0 tx=0x7f8900097c30 comp rx=0 tx=0).stop 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 shutdown_connections 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463452f80 0x55d463450a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d4634504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d463463570 0x55d4634659d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 --2- 172.21.15.94:0/1985054724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463448200 0x55d46344ffa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f88fcc00640 0 client.26018 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 >> 172.21.15.94:0/1985054724 conn(0x55d463410f80 msgr2=0x55d46344a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 shutdown_connections 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 1 -- 172.21.15.94:0/1985054724 wait complete. 2024-06-03T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.225+0000 7f8918605740 20 client.26018 trim_cache size 0 max 16384 2024-06-03T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.242+0000 7f8918605740 1 Processor -- start 2024-06-03T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.242+0000 7f8918605740 1 -- start start 2024-06-03T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.242+0000 7f8918605740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d4634437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.242+0000 7f8918605740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463449770 0x55d463454370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f8918605740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d4634548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f8918605740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b0db0 con 0x55d463448200 2024-06-03T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f8918605740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b2aa0 con 0x55d463449770 2024-06-03T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f8918605740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b14e0 con 0x55d463452f80 2024-06-03T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f890e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463449770 0x55d463454370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f890e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463449770 0x55d463454370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51846/0 (socket says 172.21.15.94:51846) 2024-06-03T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f890e000640 1 -- 172.21.15.94:0/2128444521 learned_addr learned my addr 172.21.15.94:0/2128444521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f890ea00640 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d4634437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f890ea00640 1 -- 172.21.15.94:0/2128444521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 msgr2=0x55d4634548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f890f400640 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d4634548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.243+0000 7f890ea00640 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d4634548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f890ea00640 1 -- 172.21.15.94:0/2128444521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463449770 msgr2=0x55d463454370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f890ea00640 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463449770 0x55d463454370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f890ea00640 1 -- 172.21.15.94:0/2128444521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d463448f30 con 0x55d463448200 2024-06-03T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f890e000640 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463449770 0x55d463454370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f890f400640 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d4634548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f890ea00640 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d4634437f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8900099c40 tx=0x7f8900097d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77d27a37578d8c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f88ff400640 1 -- 172.21.15.94:0/2128444521 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890009b2f0 con 0x55d463448200 2024-06-03T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f88ff400640 1 -- 172.21.15.94:0/2128444521 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890000a2f0 con 0x55d463448200 2024-06-03T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.244+0000 7f88ff400640 1 -- 172.21.15.94:0/2128444521 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89000d5240 con 0x55d463448200 2024-06-03T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.245+0000 7f8918605740 1 -- 172.21.15.94:0/2128444521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 msgr2=0x55d4634437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.245+0000 7f8918605740 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d4634437f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8900099c40 tx=0x7f8900097d60 comp rx=0 tx=0).stop 2024-06-03T01:19:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.247+0000 7f8918605740 1 -- 172.21.15.94:0/2128444521 shutdown_connections 2024-06-03T01:19:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.247+0000 7f8918605740 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463452f80 0x55d4634548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.247+0000 7f8918605740 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463449770 0x55d463454370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.247+0000 7f8918605740 1 --2- 172.21.15.94:0/2128444521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d4634437f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.247+0000 7f8918605740 1 -- 172.21.15.94:0/2128444521 >> 172.21.15.94:0/2128444521 conn(0x55d463410f80 msgr2=0x55d4634430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.248+0000 7f8918605740 1 -- 172.21.15.94:0/2128444521 shutdown_connections 2024-06-03T01:19:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.249+0000 7f8918605740 1 -- 172.21.15.94:0/2128444521 wait complete. 2024-06-03T01:19:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.251+0000 7f8918605740 1 -- start start 2024-06-03T01:19:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.255+0000 7f8918605740 1 Processor -- start 2024-06-03T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.258+0000 7f8918605740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d46345b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.258+0000 7f8918605740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d46346a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.258+0000 7f8918605740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463452f80 0x55d46346ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.258+0000 7f8918605740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b2310 con 0x55d463448200 2024-06-03T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.258+0000 7f8918605740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89101b1bb0 con 0x55d463452f80 2024-06-03T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.258+0000 7f8918605740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89101aedb0 con 0x55d463449770 2024-06-03T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.258+0000 7f8918605740 10 client.0 _ll_register_callbacks cb 0x7f8900004880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.259+0000 7f890f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463452f80 0x55d46346ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.259+0000 7f890f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463452f80 0x55d46346ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51850/0 (socket says 172.21.15.94:51850) 2024-06-03T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.259+0000 7f890f400640 1 -- 172.21.15.94:0/3812257881 learned_addr learned my addr 172.21.15.94:0/3812257881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.259+0000 7f890e000640 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d46346a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.259+0000 7f890ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d46345b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.260+0000 7f890f400640 1 -- 172.21.15.94:0/3812257881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 msgr2=0x55d46346a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.260+0000 7f890f400640 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d46346a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.260+0000 7f890f400640 1 -- 172.21.15.94:0/3812257881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 msgr2=0x55d46345b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.260+0000 7f890f400640 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d46345b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.260+0000 7f890f400640 1 -- 172.21.15.94:0/3812257881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46346b390 con 0x55d463452f80 2024-06-03T01:19:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.261+0000 7f890ea00640 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d46345b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.262+0000 7f890f400640 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463452f80 0x55d46346ae50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8908020e40 tx=0x7f89080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5851327b144ee9be server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.262+0000 7f88fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.262+0000 7f88fcc00640 1 -- 172.21.15.94:0/3812257881 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908004250 con 0x55d463452f80 2024-06-03T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.262+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890000a2f0 con 0x55d463452f80 2024-06-03T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.263+0000 7f88fcc00640 1 -- 172.21.15.94:0/3812257881 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7f89080043f0 con 0x55d463452f80 2024-06-03T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.264+0000 7f88fcc00640 1 -- 172.21.15.94:0/3812257881 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8908032050 con 0x55d463452f80 2024-06-03T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.265+0000 7f8918605740 10 client.26024 fetch_fsmap learned FSMap version 43 2024-06-03T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.265+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d463459df0 con 0x55d463452f80 2024-06-03T01:19:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.266+0000 7f88fcc00640 1 -- 172.21.15.94:0/3812257881 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890000a2f0 con 0x55d463452f80 2024-06-03T01:19:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.267+0000 7f88fcc00640 1 -- 172.21.15.94:0/3812257881 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7f8908027020 con 0x55d463452f80 2024-06-03T01:19:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.268+0000 7f8918605740 10 client.26024 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:19:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.268+0000 7f8918605740 10 client.26024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.269+0000 7f8918605740 20 client.26024 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.269+0000 7f8918605740 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d4634c8730 0x55d4634cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.269+0000 7f8918605740 4 client.26024 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:19:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.269+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4633b3ea0 con 0x55d4634c8730 2024-06-03T01:19:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.271+0000 7f890ea00640 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d4634c8730 0x55d4634cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.272+0000 7f890ea00640 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d4634c8730 0x55d4634cab50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b331eaa3578242c3 server_cookie=55a3f75d28fb7653 in_seq=0 out_seq=0 2024-06-03T01:19:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.272+0000 7f88fcc00640 10 client.26024 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.306+0000 7f88fcc00640 1 -- 172.21.15.94:0/3812257881 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d4633b3ea0 con 0x55d4634c8730 2024-06-03T01:19:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.306+0000 7f88fcc00640 10 client.26024 handle_command_reply: tid=0 2024-06-03T01:19:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.306+0000 7f8918605740 1 client.26024 shutdown 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463452f80 msgr2=0x55d46346ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463452f80 0x55d46346ae50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8908020e40 tx=0x7f89080046c0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 shutdown_connections 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55d4634c8730 0x55d4634cab50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d463452f80 0x55d46346ae50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d463449770 0x55d46346a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 --2- 172.21.15.94:0/3812257881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d463448200 0x55d46345b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f88fcc00640 0 client.26024 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 >> 172.21.15.94:0/3812257881 conn(0x55d463410f80 msgr2=0x55d4634507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 shutdown_connections 2024-06-03T01:19:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 1 -- 172.21.15.94:0/3812257881 wait complete. 2024-06-03T01:19:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.307+0000 7f8918605740 20 client.26024 trim_cache size 0 max 16384 2024-06-03T01:19:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:19:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:19:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f88f40a2cf0 msgr2=0x7f88f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f88f40a2cf0 0x7f88f40a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8908005fd0 tx=0x7f8908005e40 comp rx=0 tx=0).stop 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8910073dd0 msgr2=0x7f891019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8910073dd0 0x7f891019a200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f890000c420 tx=0x7f8900009f90 comp rx=0 tx=0).stop 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 shutdown_connections 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f88f40a2cf0 0x7f88f40a51b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f891010dfe0 0x7f891019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f89100746f0 0x7f891019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 --2- 172.21.15.94:0/1500286257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8910073dd0 0x7f891019a200 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 >> 172.21.15.94:0/1500286257 conn(0x7f89100fd2f0 msgr2=0x7f89100fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 shutdown_connections 2024-06-03T01:19:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:26.308+0000 7f890fe00640 1 -- 172.21.15.94:0/1500286257 wait complete. 2024-06-03T01:19:26.348 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:26 smithi094 ceph-mon[32119]: pgmap v281: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 371 KiB/s wr, 54 op/s 2024-06-03T01:19:26.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:26 smithi179 ceph-mon[37041]: pgmap v281: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 371 KiB/s wr, 54 op/s 2024-06-03T01:19:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:26 smithi107 ceph-mon[36622]: pgmap v281: 129 pgs: 129 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 371 KiB/s wr, 54 op/s 2024-06-03T01:19:27.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 206147 2.12 MB/sec execute 334 sec latency 27.538 ms 2024-06-03T01:19:28.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 207009 2.12 MB/sec execute 335 sec latency 16.581 ms 2024-06-03T01:19:28.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:28 smithi094 ceph-mon[32119]: pgmap v282: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 718 KiB/s wr, 93 op/s 2024-06-03T01:19:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:28 smithi179 ceph-mon[37041]: pgmap v282: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 718 KiB/s wr, 93 op/s 2024-06-03T01:19:28.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:28 smithi107 ceph-mon[36622]: pgmap v282: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 718 KiB/s wr, 93 op/s 2024-06-03T01:19:29.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 207421 2.13 MB/sec execute 336 sec latency 390.515 ms 2024-06-03T01:19:30.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:29 smithi107 ceph-mon[36622]: pgmap v283: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 60 op/s 2024-06-03T01:19:30.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:29 smithi094 ceph-mon[32119]: pgmap v283: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 60 op/s 2024-06-03T01:19:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:29 smithi179 ceph-mon[37041]: pgmap v283: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 60 op/s 2024-06-03T01:19:30.115 INFO:tasks.workunit.client.0.smithi094.stdout: 1 207421 2.12 MB/sec execute 337 sec latency 1390.618 ms 2024-06-03T01:19:31.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 208605 2.13 MB/sec execute 338 sec latency 1608.949 ms 2024-06-03T01:19:32.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 209657 2.15 MB/sec execute 339 sec latency 29.629 ms 2024-06-03T01:19:32.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:32 smithi094 ceph-mon[32119]: pgmap v284: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 60 op/s 2024-06-03T01:19:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:32 smithi179 ceph-mon[37041]: pgmap v284: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 60 op/s 2024-06-03T01:19:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:32 smithi107 ceph-mon[36622]: pgmap v284: 129 pgs: 129 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 457 KiB/s wr, 60 op/s 2024-06-03T01:19:33.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210069 2.14 MB/sec execute 340 sec latency 392.327 ms 2024-06-03T01:19:34.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210069 2.14 MB/sec execute 341 sec latency 1392.474 ms 2024-06-03T01:19:34.335 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:34 smithi107 ceph-mon[36622]: pgmap v285: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 916 KiB/s wr, 116 op/s 2024-06-03T01:19:34.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:34 smithi094 ceph-mon[32119]: pgmap v285: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 916 KiB/s wr, 116 op/s 2024-06-03T01:19:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:34 smithi179 ceph-mon[37041]: pgmap v285: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 916 KiB/s wr, 116 op/s 2024-06-03T01:19:35.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210069 2.13 MB/sec execute 342 sec latency 2392.652 ms 2024-06-03T01:19:36.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210150 2.12 MB/sec execute 343 sec latency 3328.138 ms 2024-06-03T01:19:36.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:36 smithi094 ceph-mon[32119]: pgmap v286: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 805 KiB/s wr, 95 op/s 2024-06-03T01:19:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:36 smithi179 ceph-mon[37041]: pgmap v286: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 805 KiB/s wr, 95 op/s 2024-06-03T01:19:36.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:36 smithi107 ceph-mon[36622]: pgmap v286: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 805 KiB/s wr, 95 op/s 2024-06-03T01:19:37.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210970 2.13 MB/sec execute 344 sec latency 202.990 ms 2024-06-03T01:19:38.116 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210970 2.13 MB/sec execute 345 sec latency 1203.095 ms 2024-06-03T01:19:38.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:38 smithi094 ceph-mon[32119]: pgmap v287: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-06-03T01:19:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:38 smithi179 ceph-mon[37041]: pgmap v287: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-06-03T01:19:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:38 smithi107 ceph-mon[36622]: pgmap v287: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-06-03T01:19:39.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210970 2.12 MB/sec execute 346 sec latency 2203.237 ms 2024-06-03T01:19:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 -- 172.21.15.94:0/508710154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc941047a0 msgr2=0x7fdc941087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/508710154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc941047a0 0x7fdc941087e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009900 tx=0x7fdc8002a410 comp rx=0 tx=0).stop 2024-06-03T01:19:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 -- 172.21.15.94:0/508710154 shutdown_connections 2024-06-03T01:19:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/508710154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc94108ec0 0x7fdc94110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/508710154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc941047a0 0x7fdc941087e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/508710154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc94103df0 0x7fdc941041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 -- 172.21.15.94:0/508710154 >> 172.21.15.94:0/508710154 conn(0x7fdc940ff690 msgr2=0x7fdc94101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 -- 172.21.15.94:0/508710154 shutdown_connections 2024-06-03T01:19:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.450+0000 7fdc9ba00640 1 -- 172.21.15.94:0/508710154 wait complete. 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.451+0000 7fdc9ba00640 1 Processor -- start 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.451+0000 7fdc9ba00640 1 -- start start 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.451+0000 7fdc9ba00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc94103df0 0x7fdc9419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.451+0000 7fdc9ba00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc94108ec0 0x7fdc9419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc9ba00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9419ec60 0x7fdc9419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc9ba00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94113fc0 con 0x7fdc9419ec60 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc9ba00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94113e40 con 0x7fdc94108ec0 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc9ba00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94114140 con 0x7fdc94103df0 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9419ec60 0x7fdc9419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9419ec60 0x7fdc9419f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57362/0 (socket says 172.21.15.94:57362) 2024-06-03T01:19:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 -- 172.21.15.94:0/2756129992 learned_addr learned my addr 172.21.15.94:0/2756129992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc94108ec0 0x7fdc9419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99000640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc94103df0 0x7fdc9419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 -- 172.21.15.94:0/2756129992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc94103df0 msgr2=0x7fdc9419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc94103df0 0x7fdc9419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 -- 172.21.15.94:0/2756129992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc94108ec0 msgr2=0x7fdc9419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc94108ec0 0x7fdc9419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.452+0000 7fdc99a00640 1 -- 172.21.15.94:0/2756129992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9419f750 con 0x7fdc9419ec60 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.453+0000 7fdc99a00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9419ec60 0x7fdc9419f110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00eb10 tx=0x7fdc8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.453+0000 7fdc92000640 1 -- 172.21.15.94:0/2756129992 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c004b20 con 0x7fdc9419ec60 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.453+0000 7fdc92000640 1 -- 172.21.15.94:0/2756129992 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c004cc0 con 0x7fdc9419ec60 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.453+0000 7fdc92000640 1 -- 172.21.15.94:0/2756129992 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c010760 con 0x7fdc9419ec60 2024-06-03T01:19:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.453+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9419fa40 con 0x7fdc9419ec60 2024-06-03T01:19:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.453+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc941a7200 con 0x7fdc9419ec60 2024-06-03T01:19:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.456+0000 7fdc92000640 1 -- 172.21.15.94:0/2756129992 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdc8c005020 con 0x7fdc9419ec60 2024-06-03T01:19:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.456+0000 7fdc9e853740 1 Processor -- start 2024-06-03T01:19:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc9e853740 1 -- start start 2024-06-03T01:19:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28162b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa281634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa281701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc9e853740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa280c7420 con 0x55fa281645b0 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc9e853740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa2806ec50 con 0x55fa2816ddc0 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc9e853740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa280c33e0 con 0x55fa28163040 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28162b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57372/0 (socket says 172.21.15.94:57372) 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc8be00640 1 -- 172.21.15.94:0/2567089155 learned_addr learned my addr 172.21.15.94:0/2567089155 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.457+0000 7fdc99000640 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa281634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa281701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc99000640 1 -- 172.21.15.94:0/2567089155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 msgr2=0x55fa281701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc99000640 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa281701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc99000640 1 -- 172.21.15.94:0/2567089155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 msgr2=0x55fa28162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc99000640 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28162b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc99000640 1 -- 172.21.15.94:0/2567089155 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa27ddb210 con 0x55fa28163040 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc92000640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdc780a2da0 0x7fdc780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc92000640 1 -- 172.21.15.94:0/2756129992 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7fdc8c0cff50 con 0x7fdc9419ec60 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc99000640 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa281634c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009ba0 tx=0x7fdc840117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0b2bcb70a595e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.458+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdc780a2da0 0x7fdc780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.459+0000 7fdc8a000640 1 -- 172.21.15.94:0/2567089155 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84012450 con 0x55fa28163040 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.459+0000 7fdc8a000640 1 -- 172.21.15.94:0/2567089155 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc84012ad0 con 0x55fa28163040 2024-06-03T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.459+0000 7fdc8a000640 1 -- 172.21.15.94:0/2567089155 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8400fa90 con 0x55fa28163040 2024-06-03T01:19:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.459+0000 7fdc9e853740 1 -- 172.21.15.94:0/2567089155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 msgr2=0x55fa281634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.459+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdc780a2da0 0x7fdc780a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005c40 tx=0x7fdc80005b50 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.459+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa281634c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009ba0 tx=0x7fdc840117b0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 -- 172.21.15.94:0/2567089155 shutdown_connections 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa281701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa281634c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2567089155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 -- 172.21.15.94:0/2567089155 >> 172.21.15.94:0/2567089155 conn(0x55fa2812bf80 msgr2=0x55fa2815f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 -- 172.21.15.94:0/2567089155 shutdown_connections 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 -- 172.21.15.94:0/2567089155 wait complete. 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 -- start start 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.460+0000 7fdc9e853740 1 Processor -- start 2024-06-03T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa2816adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa2816b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa2816b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc9e853740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55fa28170b90 con 0x55fa281645b0 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc9e853740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa280820b0 con 0x55fa2816ddc0 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc9e853740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa28170d10 con 0x55fa28163040 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc9e853740 10 client.0 _ll_register_callbacks cb 0x55fa2806df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa2816b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc99000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa2816adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc99a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa2816b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa2816b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57374/0 (socket says 172.21.15.94:57374) 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.461+0000 7fdc8be00640 1 -- 172.21.15.94:0/3769890478 learned_addr learned my addr 172.21.15.94:0/3769890478 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc99000640 1 -- 172.21.15.94:0/3769890478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 msgr2=0x55fa2816b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc99000640 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa2816b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc99000640 1 -- 172.21.15.94:0/3769890478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 msgr2=0x55fa2816b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc99000640 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa2816b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc99000640 1 -- 172.21.15.94:0/3769890478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa27ea7620 con 0x55fa28163040 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc99000640 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa2816adf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc8401c040 tx=0x7fdc84012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeace42b76bef770 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc7ea00640 1 -- 172.21.15.94:0/3769890478 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84028070 con 0x55fa28163040 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc7ea00640 1 -- 172.21.15.94:0/3769890478 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7fdc84004020 con 0x55fa28163040 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc84012ad0 con 0x55fa28163040 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc7ea00640 1 -- 172.21.15.94:0/3769890478 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdc84031050 con 0x55fa28163040 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc9e853740 10 client.26047 fetch_fsmap learned FSMap version 43 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.462+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa2817cf00 con 0x55fa28163040 2024-06-03T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc7ea00640 1 -- 172.21.15.94:0/3769890478 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc84012ad0 con 0x55fa28163040 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc7ea00640 1 -- 172.21.15.94:0/3769890478 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fdc8402b020 con 0x55fa28163040 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc9e853740 10 client.26047 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc9e853740 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc9e853740 20 client.26047 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc9e853740 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa2817e360 0x55fa28180750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc9e853740 4 client.26047 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa280d4b50 con 0x55fa2817e360 2024-06-03T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.463+0000 7fdc8be00640 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa2817e360 0x55fa28180750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.464+0000 7fdc7ea00640 10 client.26047 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.464+0000 7fdc8be00640 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa2817e360 0x55fa28180750 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=414d9efc98cc7412 server_cookie=37cf611e40a56add in_seq=0 out_seq=0 2024-06-03T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc7ea00640 1 -- 172.21.15.94:0/3769890478 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa280d4b50 con 0x55fa2817e360 2024-06-03T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc7ea00640 10 client.26047 handle_command_reply: tid=0 2024-06-03T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 client.26047 shutdown 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 msgr2=0x55fa2816adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa2816adf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc8401c040 tx=0x7fdc84012eb0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 shutdown_connections 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa2817e360 0x55fa28180750 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa2816ddc0 0x55fa2816b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa2816b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc7ea00640 0 client.26047 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 --2- 172.21.15.94:0/3769890478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa28163040 0x55fa2816adf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 >> 172.21.15.94:0/3769890478 conn(0x55fa2812bf80 msgr2=0x55fa281654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 shutdown_connections 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.465+0000 7fdc9e853740 1 -- 172.21.15.94:0/3769890478 wait complete. 2024-06-03T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.466+0000 7fdc9e853740 20 client.26047 trim_cache size 0 max 16384 2024-06-03T01:19:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 Processor -- start 2024-06-03T01:19:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 -- start start 2024-06-03T01:19:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa28163040 0x55fa28169d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa281645b0 0x55fa2815e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28185de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941b1010 con 0x55fa28163040 2024-06-03T01:19:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941b2d00 con 0x55fa281645b0 2024-06-03T01:19:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.482+0000 7fdc9e853740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941b1740 con 0x55fa2816ddc0 2024-06-03T01:19:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa281645b0 0x55fa2815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa281645b0 0x55fa2815e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:55152/0 (socket says 172.21.15.94:55152) 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc99000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa28163040 0x55fa28169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 -- 172.21.15.94:0/433606928 learned_addr learned my addr 172.21.15.94:0/433606928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc99a00640 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28185de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 -- 172.21.15.94:0/433606928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 msgr2=0x55fa28185de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28185de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 -- 172.21.15.94:0/433606928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa28163040 msgr2=0x55fa28169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa28163040 0x55fa28169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 -- 172.21.15.94:0/433606928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa28169340 con 0x55fa281645b0 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc99000640 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa28163040 0x55fa28169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.483+0000 7fdc8be00640 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa281645b0 0x55fa2815e690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc8002aa00 tx=0x7fdc8003b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4c69da0842eec2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc89600640 1 -- 172.21.15.94:0/433606928 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80035040 con 0x55fa281645b0 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc89600640 1 -- 172.21.15.94:0/433606928 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc80004c60 con 0x55fa281645b0 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc89600640 1 -- 172.21.15.94:0/433606928 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80032e30 con 0x55fa281645b0 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 -- 172.21.15.94:0/433606928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa281645b0 msgr2=0x55fa2815e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa281645b0 0x55fa2815e690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc8002aa00 tx=0x7fdc8003b000 comp rx=0 tx=0).stop 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 -- 172.21.15.94:0/433606928 shutdown_connections 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28185de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa281645b0 0x55fa2815e690 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 --2- 172.21.15.94:0/433606928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa28163040 0x55fa28169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 -- 172.21.15.94:0/433606928 >> 172.21.15.94:0/433606928 conn(0x55fa2812bf80 msgr2=0x55fa2815df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 -- 172.21.15.94:0/433606928 shutdown_connections 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.484+0000 7fdc9e853740 1 -- 172.21.15.94:0/433606928 wait complete. 2024-06-03T01:19:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.485+0000 7fdc9e853740 1 -- start start 2024-06-03T01:19:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.485+0000 7fdc9e853740 1 Processor -- start 2024-06-03T01:19:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.485+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa28163040 0x55fa28176200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.485+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28176740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.485+0000 7fdc9e853740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28167660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc9e853740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941b2570 con 0x55fa281645b0 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc9e853740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941b1e10 con 0x55fa28163040 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc9e853740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc941af010 con 0x55fa2816ddc0 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc9e853740 10 client.0 _ll_register_callbacks cb 0x7fdc8c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28176740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28176740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57386/0 (socket says 172.21.15.94:57386) 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 -- 172.21.15.94:0/2107509007 learned_addr learned my addr 172.21.15.94:0/2107509007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc99a00640 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc99000640 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa28163040 0x55fa28176200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 -- 172.21.15.94:0/2107509007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 msgr2=0x55fa28167660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28167660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 -- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa28163040 msgr2=0x55fa28176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa28163040 0x55fa28176200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.486+0000 7fdc8be00640 1 -- 172.21.15.94:0/2107509007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa28167e90 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc8be00640 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28176740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005310 tx=0x7fdc80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623ad87281c6d59a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc7ea00640 1 -- 172.21.15.94:0/2107509007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80063060 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc7ea00640 1 -- 172.21.15.94:0/2107509007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7fdc800325c0 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc80004c60 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc7ea00640 1 -- 172.21.15.94:0/2107509007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdc8006a050 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc9e853740 10 client.15930 fetch_fsmap learned FSMap version 43 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa281f10f0 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.487+0000 7fdc7ea00640 1 -- 172.21.15.94:0/2107509007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc80004c60 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc7ea00640 1 -- 172.21.15.94:0/2107509007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 43) ==== 8398+0+0 (secure 0 0 0) 0x7fdc8005d070 con 0x55fa281645b0 2024-06-03T01:19:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc9e853740 10 client.15930 fetch_fsmap finished waiting for FSMap version 43 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc9e853740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc9e853740 20 client.15930 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa281f1850 0x55fa281f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc9e853740 4 client.15930 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa280ceea0 con 0x55fa281f1850 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.488+0000 7fdc99000640 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa281f1850 0x55fa281f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.489+0000 7fdc99000640 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa281f1850 0x55fa281f3c40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e2f6efdffc03d5 server_cookie=28802f7d4d5b5df9 in_seq=0 out_seq=0 2024-06-03T01:19:39.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.489+0000 7fdc7ea00640 10 client.15930 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.513+0000 7fdc7ea00640 1 -- 172.21.15.94:0/2107509007 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fa280ceea0 con 0x55fa281f1850 2024-06-03T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.513+0000 7fdc7ea00640 10 client.15930 handle_command_reply: tid=0 2024-06-03T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.513+0000 7fdc9e853740 1 client.15930 shutdown 2024-06-03T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.513+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 msgr2=0x55fa28176740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.513+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28176740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005310 tx=0x7fdc80000bc0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 shutdown_connections 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa2816ddc0 0x55fa28167660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa281645b0 0x55fa28176740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55fa281f1850 0x55fa281f3c40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 --2- 172.21.15.94:0/2107509007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa28163040 0x55fa28176200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc7ea00640 0 client.15930 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 >> 172.21.15.94:0/2107509007 conn(0x55fa2812bf80 msgr2=0x55fa281654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 shutdown_connections 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 1 -- 172.21.15.94:0/2107509007 wait complete. 2024-06-03T01:19:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.514+0000 7fdc9e853740 20 client.15930 trim_cache size 0 max 16384 2024-06-03T01:19:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:19:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:19:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdc780a2da0 msgr2=0x7fdc780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdc780a2da0 0x7fdc780a5260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005c40 tx=0x7fdc80005b50 comp rx=0 tx=0).stop 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9419ec60 msgr2=0x7fdc9419f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9419ec60 0x7fdc9419f110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00eb10 tx=0x7fdc8c00c2d0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 shutdown_connections 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdc780a2da0 0x7fdc780a5260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9419ec60 0x7fdc9419f110 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc94108ec0 0x7fdc9419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 --2- 172.21.15.94:0/2756129992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc94103df0 0x7fdc9419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 >> 172.21.15.94:0/2756129992 conn(0x7fdc940ff690 msgr2=0x7fdc94100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 shutdown_connections 2024-06-03T01:19:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:39.515+0000 7fdc9ba00640 1 -- 172.21.15.94:0/2756129992 wait complete. 2024-06-03T01:19:40.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 210970 2.11 MB/sec execute 347 sec latency 3203.328 ms 2024-06-03T01:19:40.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:40 smithi094 ceph-mon[32119]: pgmap v288: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 97 op/s 2024-06-03T01:19:40.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:40 smithi179 ceph-mon[37041]: pgmap v288: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 97 op/s 2024-06-03T01:19:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:40 smithi107 ceph-mon[36622]: pgmap v288: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 KiB/s wr, 97 op/s 2024-06-03T01:19:41.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 212192 2.13 MB/sec execute 348 sec latency 3404.127 ms 2024-06-03T01:19:42.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213230 2.14 MB/sec execute 349 sec latency 32.087 ms 2024-06-03T01:19:42.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:42 smithi094 ceph-mon[32119]: pgmap v289: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 897 KiB/s wr, 97 op/s 2024-06-03T01:19:42.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:42 smithi179 ceph-mon[37041]: pgmap v289: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 897 KiB/s wr, 97 op/s 2024-06-03T01:19:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:42 smithi107 ceph-mon[36622]: pgmap v289: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 897 KiB/s wr, 97 op/s 2024-06-03T01:19:43.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213485 2.14 MB/sec execute 350 sec latency 628.072 ms 2024-06-03T01:19:43.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:43 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:43.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:43 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:43.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:43 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:19:44.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213485 2.13 MB/sec execute 351 sec latency 1628.181 ms 2024-06-03T01:19:44.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:44 smithi094 ceph-mon[32119]: pgmap v290: 129 pgs: 129 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-06-03T01:19:44.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:44 smithi179 ceph-mon[37041]: pgmap v290: 129 pgs: 129 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-06-03T01:19:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:44 smithi107 ceph-mon[36622]: pgmap v290: 129 pgs: 129 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-06-03T01:19:45.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213485 2.12 MB/sec execute 352 sec latency 2628.347 ms 2024-06-03T01:19:45.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:45 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:19:45.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:45 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:19:45.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:45 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:19:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:45 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:19:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:45 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:19:45.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:45 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:19:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:45 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:19:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:45 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:19:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:45 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:19:46.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213540 2.12 MB/sec execute 353 sec latency 3565.918 ms 2024-06-03T01:19:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:46 smithi107 ceph-mon[36622]: pgmap v291: 129 pgs: 129 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 803 KiB/s wr, 81 op/s 2024-06-03T01:19:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:46 smithi094 ceph-mon[32119]: pgmap v291: 129 pgs: 129 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 803 KiB/s wr, 81 op/s 2024-06-03T01:19:46.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:46 smithi179 ceph-mon[37041]: pgmap v291: 129 pgs: 129 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 803 KiB/s wr, 81 op/s 2024-06-03T01:19:47.117 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213609 2.11 MB/sec execute 354 sec latency 897.917 ms 2024-06-03T01:19:48.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213609 2.11 MB/sec execute 355 sec latency 1898.073 ms 2024-06-03T01:19:48.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:48 smithi107 ceph-mon[36622]: pgmap v292: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 949 KiB/s wr, 103 op/s 2024-06-03T01:19:48.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:48 smithi094 ceph-mon[32119]: pgmap v292: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 949 KiB/s wr, 103 op/s 2024-06-03T01:19:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:48 smithi179 ceph-mon[37041]: pgmap v292: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 949 KiB/s wr, 103 op/s 2024-06-03T01:19:49.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213609 2.10 MB/sec execute 356 sec latency 2898.219 ms 2024-06-03T01:19:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:49 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:19:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:49 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:19:49.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:49 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:19:50.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213609 2.09 MB/sec execute 357 sec latency 3898.384 ms 2024-06-03T01:19:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:50 smithi094 ceph-mon[32119]: pgmap v293: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-06-03T01:19:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:50 smithi179 ceph-mon[37041]: pgmap v293: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-06-03T01:19:50.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:50 smithi107 ceph-mon[36622]: pgmap v293: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-06-03T01:19:51.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 213690 2.09 MB/sec execute 358 sec latency 4833.140 ms 2024-06-03T01:19:52.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 214512 2.10 MB/sec execute 359 sec latency 213.321 ms 2024-06-03T01:19:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:52 smithi094 ceph-mon[32119]: pgmap v294: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-06-03T01:19:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:52 smithi179 ceph-mon[37041]: pgmap v294: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-06-03T01:19:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 -- 172.21.15.94:0/420110814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 msgr2=0x7f915410b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 --2- 172.21.15.94:0/420110814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 0x7f915410b780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f913c009860 tx=0x7f913c02a3d0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 -- 172.21.15.94:0/420110814 shutdown_connections 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 --2- 172.21.15.94:0/420110814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 0x7f915410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 --2- 172.21.15.94:0/420110814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 0x7f915410b780 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 --2- 172.21.15.94:0/420110814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 0x7f9154102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 -- 172.21.15.94:0/420110814 >> 172.21.15.94:0/420110814 conn(0x7f91540fc1c0 msgr2=0x7f91540fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 -- 172.21.15.94:0/420110814 shutdown_connections 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.660+0000 7f9159c00640 1 -- 172.21.15.94:0/420110814 wait complete. 2024-06-03T01:19:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.661+0000 7f9159c00640 1 Processor -- start 2024-06-03T01:19:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159c00640 1 -- start start 2024-06-03T01:19:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 0x7f91541005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 0x7f9154100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 0x7f9154101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9154111ba0 con 0x7f915410bda0 2024-06-03T01:19:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9154111a20 con 0x7f9154102980 2024-06-03T01:19:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9154111d20 con 0x7f91541032a0 2024-06-03T01:19:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 0x7f9154101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9153e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 0x7f91541005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 0x7f9154101050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50562/0 (socket says 172.21.15.94:50562) 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9153e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 0x7f91541005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:34556/0 (socket says 172.21.15.94:34556) 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.662+0000 7f9159200640 1 -- 172.21.15.94:0/3561449447 learned_addr learned my addr 172.21.15.94:0/3561449447 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f914be00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 0x7f9154100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9153e00640 1 -- 172.21.15.94:0/3561449447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 msgr2=0x7f9154100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9153e00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 0x7f9154100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9153e00640 1 -- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 msgr2=0x7f9154101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9153e00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 0x7f9154101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9153e00640 1 -- 172.21.15.94:0/3561449447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91541a6580 con 0x7f9154102980 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9159200640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 0x7f9154101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9153e00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 0x7f91541005d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.663+0000 7f9151600640 1 -- 172.21.15.94:0/3561449447 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914400ccc0 con 0x7f9154102980 2024-06-03T01:19:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.664+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91541a6870 con 0x7f9154102980 2024-06-03T01:19:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.664+0000 7f9151600640 1 -- 172.21.15.94:0/3561449447 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9144010910 con 0x7f9154102980 2024-06-03T01:19:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.664+0000 7f9151600640 1 -- 172.21.15.94:0/3561449447 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144018960 con 0x7f9154102980 2024-06-03T01:19:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.664+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a6e10 con 0x7f9154102980 2024-06-03T01:19:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 Processor -- start 2024-06-03T01:19:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f9151600640 1 -- 172.21.15.94:0/3561449447 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9144010ac0 con 0x7f9154102980 2024-06-03T01:19:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 -- start start 2024-06-03T01:19:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x556434987cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x556434988680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x556434995370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5564348ec420 con 0x556434992f80 2024-06-03T01:19:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556434893c50 con 0x556434988200 2024-06-03T01:19:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.667+0000 7f915e1b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5564348e83e0 con 0x556434989770 2024-06-03T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x556434988680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f914be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x556434995370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x556434988680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:34558/0 (socket says 172.21.15.94:34558) 2024-06-03T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f914be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x556434995370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50572/0 (socket says 172.21.15.94:50572) 2024-06-03T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 -- 172.21.15.94:0/549431576 learned_addr learned my addr 172.21.15.94:0/549431576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9159200640 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x556434987cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 -- 172.21.15.94:0/549431576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 msgr2=0x556434987cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x556434987cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 -- 172.21.15.94:0/549431576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 msgr2=0x556434995370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x556434995370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 -- 172.21.15.94:0/549431576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556434600210 con 0x556434988200 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9159200640 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x556434987cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9151600640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.668+0000 7f9153e00640 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x556434988680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f91440d3c90 tx=0x7f91440d85e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d7e42c022a933a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.669+0000 7f9151600640 1 -- 172.21.15.94:0/3561449447 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7f91440cfb20 con 0x7f9154102980 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.669+0000 7f9149600640 1 -- 172.21.15.94:0/549431576 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91440d5050 con 0x556434988200 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.669+0000 7f9159200640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.669+0000 7f9149600640 1 -- 172.21.15.94:0/549431576 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f914409bcd0 con 0x556434988200 2024-06-03T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.669+0000 7f9149600640 1 -- 172.21.15.94:0/549431576 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914409baa0 con 0x556434988200 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 -- 172.21.15.94:0/549431576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 msgr2=0x556434988680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x556434988680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f91440d3c90 tx=0x7f91440d85e0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f9159200640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f914c009a70 tx=0x7f914c0095c0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 -- 172.21.15.94:0/549431576 shutdown_connections 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x556434995370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x556434988680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 --2- 172.21.15.94:0/549431576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x556434987cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 -- 172.21.15.94:0/549431576 >> 172.21.15.94:0/549431576 conn(0x556434950f80 msgr2=0x5564349847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 -- 172.21.15.94:0/549431576 shutdown_connections 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 -- 172.21.15.94:0/549431576 wait complete. 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 -- start start 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.670+0000 7f915e1b6740 1 Processor -- start 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434988200 0x55643498ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434989770 0x556434990490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f915e1b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556434995cf0 con 0x556434992f80 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f915e1b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5564348a70b0 con 0x556434989770 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f915e1b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556434995e70 con 0x556434988200 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f915e1b6740 10 client.0 _ll_register_callbacks cb 0x556434892f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349909d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50574/0 (socket says 172.21.15.94:50574) 2024-06-03T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f9159200640 1 -- 172.21.15.94:0/744688431 learned_addr learned my addr 172.21.15.94:0/744688431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f914be00640 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434989770 0x556434990490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.671+0000 7f9153e00640 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434988200 0x55643498ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9159200640 1 -- 172.21.15.94:0/744688431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434988200 msgr2=0x55643498ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9159200640 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434988200 0x55643498ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9159200640 1 -- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434989770 msgr2=0x556434990490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9159200640 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434989770 0x556434990490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9159200640 1 -- 172.21.15.94:0/744688431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564346cc620 con 0x556434992f80 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9159200640 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349909d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f914c00bfc0 tx=0x7f914c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2518b8965fcc869 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9142000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9142000640 1 -- 172.21.15.94:0/744688431 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c0255f0 con 0x556434992f80 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f9142000640 1 -- 172.21.15.94:0/744688431 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7f914c025790 con 0x556434992f80 2024-06-03T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.672+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f914409bcd0 con 0x556434992f80 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f9142000640 1 -- 172.21.15.94:0/744688431 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f914c025990 con 0x556434992f80 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 10 client.15942 fetch_fsmap learned FSMap version 44 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5564349a2080 con 0x556434992f80 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f9142000640 1 -- 172.21.15.94:0/744688431 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f914409bcd0 con 0x556434992f80 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f9142000640 1 -- 172.21.15.94:0/744688431 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 44) ==== 8394+0+0 (secure 0 0 0) 0x7f914c00b040 con 0x556434992f80 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 10 client.15942 fetch_fsmap finished waiting for FSMap version 44 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 20 client.15942 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5564349a3650 0x5564349a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 4 client.15942 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.673+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564348f9b50 con 0x5564349a3650 2024-06-03T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.674+0000 7f9153e00640 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5564349a3650 0x5564349a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.674+0000 7f9153e00640 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5564349a3650 0x5564349a5a90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8569412f6fbdf3 server_cookie=63d2b611ae17517 in_seq=0 out_seq=0 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.674+0000 7f9142000640 10 client.15942 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f9142000640 1 -- 172.21.15.94:0/744688431 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5564348f9b50 con 0x5564349a3650 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f9142000640 10 client.15942 handle_command_reply: tid=0 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 client.15942 shutdown 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 msgr2=0x5564349909d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349909d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f914c00bfc0 tx=0x7f914c004ab0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 shutdown_connections 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349909d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5564349a3650 0x5564349a5a90 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434989770 0x556434990490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 --2- 172.21.15.94:0/744688431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434988200 0x55643498ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f9142000640 0 client.15942 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 >> 172.21.15.94:0/744688431 conn(0x556434950f80 msgr2=0x55643498a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 shutdown_connections 2024-06-03T01:19:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.675+0000 7f915e1b6740 1 -- 172.21.15.94:0/744688431 wait complete. 2024-06-03T01:19:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.676+0000 7f915e1b6740 20 client.15942 trim_cache size 0 max 16384 2024-06-03T01:19:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 Processor -- start 2024-06-03T01:19:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 -- start start 2024-06-03T01:19:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643498f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643498a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b0f40 con 0x556434992f80 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b2c00 con 0x556434988200 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.692+0000 7f915e1b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b1640 con 0x556434989770 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9153e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643498f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50586/0 (socket says 172.21.15.94:50586) 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9159200640 1 -- 172.21.15.94:0/1694365517 learned_addr learned my addr 172.21.15.94:0/1694365517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f914be00640 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643498a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9153e00640 1 -- 172.21.15.94:0/1694365517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 msgr2=0x55643498a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9153e00640 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643498a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9153e00640 1 -- 172.21.15.94:0/1694365517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 msgr2=0x5564349840f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9153e00640 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349840f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9153e00640 1 -- 172.21.15.94:0/1694365517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55643498e4a0 con 0x556434988200 2024-06-03T01:19:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9159200640 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:19:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.693+0000 7f9153e00640 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643498f0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55643498aed0 tx=0x7f914400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e57996078d00e5a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f9148c00640 1 -- 172.21.15.94:0/1694365517 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91440d5070 con 0x556434988200 2024-06-03T01:19:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f9148c00640 1 -- 172.21.15.94:0/1694365517 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f914409ffd0 con 0x556434988200 2024-06-03T01:19:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f9148c00640 1 -- 172.21.15.94:0/1694365517 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914409b630 con 0x556434988200 2024-06-03T01:19:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 -- 172.21.15.94:0/1694365517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 msgr2=0x55643498f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643498f0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55643498aed0 tx=0x7f914400cf60 comp rx=0 tx=0).stop 2024-06-03T01:19:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 -- 172.21.15.94:0/1694365517 shutdown_connections 2024-06-03T01:19:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643498a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 --2- 172.21.15.94:0/1694365517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643498f0b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 -- 172.21.15.94:0/1694365517 >> 172.21.15.94:0/1694365517 conn(0x556434950f80 msgr2=0x556434989d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 -- 172.21.15.94:0/1694365517 shutdown_connections 2024-06-03T01:19:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.694+0000 7f915e1b6740 1 -- 172.21.15.94:0/1694365517 wait complete. 2024-06-03T01:19:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 -- start start 2024-06-03T01:19:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 Processor -- start 2024-06-03T01:19:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643499b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643499b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b2470 con 0x556434992f80 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b1d10 con 0x556434988200 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f91541aef40 con 0x556434989770 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.695+0000 7f915e1b6740 10 client.0 _ll_register_callbacks cb 0x7f914c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50594/0 (socket says 172.21.15.94:50594) 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 -- 172.21.15.94:0/165958621 learned_addr learned my addr 172.21.15.94:0/165958621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9153e00640 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643499b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f914be00640 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643499b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 -- 172.21.15.94:0/165958621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 msgr2=0x55643499b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643499b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 -- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 msgr2=0x55643499b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643499b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 -- 172.21.15.94:0/165958621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556434996f10 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9159200640 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349966e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f914c02e3b0 tx=0x7f914c02e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4200b1666efbc3c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.696+0000 7f9142000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f9142000640 1 -- 172.21.15.94:0/165958621 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c005c70 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f914409ffd0 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f9142000640 1 -- 172.21.15.94:0/165958621 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9475+0+0 (secure 0 0 0) 0x7f914c0253f0 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f9142000640 1 -- 172.21.15.94:0/165958621 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f914c047050 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f915e1b6740 10 client.15948 fetch_fsmap learned FSMap version 44 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556434a0de80 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f9142000640 1 -- 172.21.15.94:0/165958621 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f914409ffd0 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.697+0000 7f9142000640 1 -- 172.21.15.94:0/165958621 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 44) ==== 8394+0+0 (secure 0 0 0) 0x7f914c019460 con 0x556434992f80 2024-06-03T01:19:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f915e1b6740 10 client.15948 fetch_fsmap finished waiting for FSMap version 44 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f915e1b6740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f915e1b6740 20 client.15948 populate_metadata read hostname 'smithi094' 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f915e1b6740 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556434a0e6e0 0x556434a10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f915e1b6740 4 client.15948 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564348f3ea0 con 0x556434a0e6e0 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f9153e00640 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556434a0e6e0 0x556434a10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f9153e00640 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556434a0e6e0 0x556434a10b00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8411809323da42 server_cookie=7a689c0bea190546 in_seq=0 out_seq=0 2024-06-03T01:19:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.698+0000 7f9142000640 10 client.15948 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.721+0000 7f9142000640 1 -- 172.21.15.94:0/165958621 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5564348f3ea0 con 0x556434a0e6e0 2024-06-03T01:19:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.721+0000 7f9142000640 10 client.15948 handle_command_reply: tid=0 2024-06-03T01:19:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 client.15948 shutdown 2024-06-03T01:19:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 msgr2=0x5564349966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349966e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f914c02e3b0 tx=0x7f914c02e660 comp rx=0 tx=0).stop 2024-06-03T01:19:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 shutdown_connections 2024-06-03T01:19:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556434992f80 0x5564349966e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556434989770 0x55643499b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x556434a0e6e0 0x556434a10b00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 --2- 172.21.15.94:0/165958621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556434988200 0x55643499b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f9142000640 0 client.15948 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 >> 172.21.15.94:0/165958621 conn(0x556434950f80 msgr2=0x5564349907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 shutdown_connections 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 1 -- 172.21.15.94:0/165958621 wait complete. 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.722+0000 7f915e1b6740 20 client.15948 trim_cache size 0 max 16384 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:19:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:19:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:19:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 msgr2=0x7f91380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f914c009a70 tx=0x7f914c0095c0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 msgr2=0x7f91541005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:19:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 0x7f91541005d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400eeb0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 shutdown_connections 2024-06-03T01:19:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f915410bda0 0x7f9154101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541032a0 0x7f9154100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 --2- 172.21.15.94:0/3561449447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154102980 0x7f91541005d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:19:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 >> 172.21.15.94:0/3561449447 conn(0x7f91540fc1c0 msgr2=0x7f91540fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:19:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 shutdown_connections 2024-06-03T01:19:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:19:52.723+0000 7f9159c00640 1 -- 172.21.15.94:0/3561449447 wait complete. 2024-06-03T01:19:52.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:52 smithi107 ceph-mon[36622]: pgmap v294: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-06-03T01:19:53.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 214512 2.09 MB/sec execute 360 sec latency 1213.420 ms 2024-06-03T01:19:54.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 214512 2.08 MB/sec execute 361 sec latency 2213.513 ms 2024-06-03T01:19:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:54 smithi094 ceph-mon[32119]: pgmap v295: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 619 KiB/s wr, 83 op/s 2024-06-03T01:19:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:54 smithi179 ceph-mon[37041]: pgmap v295: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 619 KiB/s wr, 83 op/s 2024-06-03T01:19:54.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:54 smithi107 ceph-mon[36622]: pgmap v295: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 619 KiB/s wr, 83 op/s 2024-06-03T01:19:55.118 INFO:tasks.workunit.client.0.smithi094.stdout: 1 214512 2.08 MB/sec execute 362 sec latency 3213.660 ms 2024-06-03T01:19:56.119 INFO:tasks.workunit.client.0.smithi094.stdout: 1 215655 2.09 MB/sec execute 363 sec latency 3416.489 ms 2024-06-03T01:19:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:56 smithi094 ceph-mon[32119]: pgmap v296: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-06-03T01:19:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:56 smithi179 ceph-mon[37041]: pgmap v296: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-06-03T01:19:56.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:56 smithi107 ceph-mon[36622]: pgmap v296: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-06-03T01:19:57.119 INFO:tasks.workunit.client.0.smithi094.stdout: 1 216731 2.10 MB/sec execute 364 sec latency 22.262 ms 2024-06-03T01:19:58.119 INFO:tasks.workunit.client.0.smithi094.stdout: 1 217145 2.10 MB/sec execute 365 sec latency 270.028 ms 2024-06-03T01:19:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:58 smithi094 ceph-mon[32119]: pgmap v297: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-06-03T01:19:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:58 smithi179 ceph-mon[37041]: pgmap v297: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-06-03T01:19:58.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:58 smithi107 ceph-mon[36622]: pgmap v297: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-06-03T01:19:59.119 INFO:tasks.workunit.client.0.smithi094.stdout: 1 217145 2.09 MB/sec execute 366 sec latency 1270.166 ms 2024-06-03T01:19:59.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:19:59 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:19:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:19:59 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:19:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:19:59 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:00.119 INFO:tasks.workunit.client.0.smithi094.stdout: 1 217145 2.09 MB/sec execute 367 sec latency 2270.315 ms 2024-06-03T01:20:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:00 smithi094 ceph-mon[32119]: pgmap v298: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 344 KiB/s wr, 56 op/s 2024-06-03T01:20:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:00 smithi094 ceph-mon[32119]: overall HEALTH_OK 2024-06-03T01:20:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:00 smithi179 ceph-mon[37041]: pgmap v298: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 344 KiB/s wr, 56 op/s 2024-06-03T01:20:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:00 smithi179 ceph-mon[37041]: overall HEALTH_OK 2024-06-03T01:20:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:00 smithi107 ceph-mon[36622]: pgmap v298: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 344 KiB/s wr, 56 op/s 2024-06-03T01:20:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:00 smithi107 ceph-mon[36622]: overall HEALTH_OK 2024-06-03T01:20:01.119 INFO:tasks.workunit.client.0.smithi094.stdout: 1 217439 2.09 MB/sec execute 368 sec latency 2459.869 ms 2024-06-03T01:20:02.120 INFO:tasks.workunit.client.0.smithi094.stdout: 1 217610 2.08 MB/sec execute 369 sec latency 100.422 ms 2024-06-03T01:20:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:02 smithi094 ceph-mon[32119]: pgmap v299: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 344 KiB/s wr, 56 op/s 2024-06-03T01:20:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:02 smithi179 ceph-mon[37041]: pgmap v299: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 344 KiB/s wr, 56 op/s 2024-06-03T01:20:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:02 smithi107 ceph-mon[36622]: pgmap v299: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 344 KiB/s wr, 56 op/s 2024-06-03T01:20:03.120 INFO:tasks.workunit.client.0.smithi094.stdout: 1 218114 2.09 MB/sec execute 370 sec latency 534.034 ms 2024-06-03T01:20:04.120 INFO:tasks.workunit.client.0.smithi094.stdout: 1 219395 2.10 MB/sec execute 371 sec latency 537.650 ms 2024-06-03T01:20:04.604 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:04 smithi107 ceph-mon[36622]: pgmap v300: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 851 KiB/s wr, 123 op/s 2024-06-03T01:20:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:04 smithi179 ceph-mon[37041]: pgmap v300: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 851 KiB/s wr, 123 op/s 2024-06-03T01:20:04.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:04 smithi094 ceph-mon[32119]: pgmap v300: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 851 KiB/s wr, 123 op/s 2024-06-03T01:20:05.121 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220316 2.11 MB/sec execute 372 sec latency 33.581 ms 2024-06-03T01:20:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:05 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:05 smithi179 ceph-mon[37041]: osdmap e108: 12 total, 12 up, 12 in 2024-06-03T01:20:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:05 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:05 smithi107 ceph-mon[36622]: osdmap e108: 12 total, 12 up, 12 in 2024-06-03T01:20:05.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:05 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:05.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:05 smithi094 ceph-mon[32119]: osdmap e108: 12 total, 12 up, 12 in 2024-06-03T01:20:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.869+0000 7faaafa00640 1 -- 172.21.15.94:0/2676606387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810f4b0 msgr2=0x7faaa8117080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.869+0000 7faaafa00640 1 --2- 172.21.15.94:0/2676606387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810f4b0 0x7faaa8117080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009ca0 tx=0x7faaa002a8c0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.869+0000 7faaafa00640 1 -- 172.21.15.94:0/2676606387 shutdown_connections 2024-06-03T01:20:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.869+0000 7faaafa00640 1 --2- 172.21.15.94:0/2676606387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810f4b0 0x7faaa8117080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.869+0000 7faaafa00640 1 --2- 172.21.15.94:0/2676606387 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa810ade0 0x7faaa810edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.869+0000 7faaafa00640 1 --2- 172.21.15.94:0/2676606387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa810a430 0x7faaa810a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.869+0000 7faaafa00640 1 -- 172.21.15.94:0/2676606387 >> 172.21.15.94:0/2676606387 conn(0x7faaa806eb20 msgr2=0x7faaa81089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.870+0000 7faaafa00640 1 -- 172.21.15.94:0/2676606387 shutdown_connections 2024-06-03T01:20:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.870+0000 7faaafa00640 1 -- 172.21.15.94:0/2676606387 wait complete. 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 Processor -- start 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 -- start start 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810a430 0x7faaa806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa810f4b0 0x7faaa806b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa806be90 0x7faaa81aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaa811a550 con 0x7faaa810f4b0 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faaa811a3d0 con 0x7faaa806be90 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaafa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faaa811a6d0 con 0x7faaa810a430 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa806be90 0x7faaa81aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaa7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa810f4b0 0x7faaa806b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaad000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810a430 0x7faaa806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa806be90 0x7faaa81aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:43236/0 (socket says 172.21.15.94:43236) 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.871+0000 7faaada00640 1 -- 172.21.15.94:0/224066222 learned_addr learned my addr 172.21.15.94:0/224066222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaad000640 1 -- 172.21.15.94:0/224066222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa806be90 msgr2=0x7faaa81aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaad000640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa806be90 0x7faaa81aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaad000640 1 -- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa810f4b0 msgr2=0x7faaa806b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaad000640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa810f4b0 0x7faaa806b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaad000640 1 -- 172.21.15.94:0/224066222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa81af200 con 0x7faaa810a430 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaa7e00640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa810f4b0 0x7faaa806b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaad000640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810a430 0x7faaa806b410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00e9f0 tx=0x7faa9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaa5600640 1 -- 172.21.15.94:0/224066222 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00ccc0 con 0x7faaa810a430 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaa5600640 1 -- 172.21.15.94:0/224066222 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9c010910 con 0x7faaa810a430 2024-06-03T01:20:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.872+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa81af4f0 con 0x7faaa810a430 2024-06-03T01:20:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.873+0000 7faaa5600640 1 -- 172.21.15.94:0/224066222 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c018960 con 0x7faaa810a430 2024-06-03T01:20:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.873+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa81afa90 con 0x7faaa810a430 2024-06-03T01:20:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 Processor -- start 2024-06-03T01:20:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 -- start start 2024-06-03T01:20:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faaa5600640 1 -- 172.21.15.94:0/224066222 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7faa9c010ac0 con 0x7faaa810a430 2024-06-03T01:20:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bca5b0 0x561b60bc8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bc94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561b60b2d420 con 0x561b60bc9040 2024-06-03T01:20:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561b60ad4c50 con 0x561b60bca5b0 2024-06-03T01:20:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.876+0000 7faab27fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561b60b293e0 con 0x561b60bd3dc0 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaad000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bc94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bca5b0 0x561b60bc8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bca5b0 0x561b60bc8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:43252/0 (socket says 172.21.15.94:43252) 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 -- 172.21.15.94:0/1906528340 learned_addr learned my addr 172.21.15.94:0/1906528340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaa7e00640 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 -- 172.21.15.94:0/1906528340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 msgr2=0x561b60bd61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 -- 172.21.15.94:0/1906528340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 msgr2=0x561b60bc94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bc94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 -- 172.21.15.94:0/1906528340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b60841210 con 0x561b60bca5b0 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.877+0000 7faaada00640 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bca5b0 0x561b60bc8b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faaa002f530 tx=0x7faaa0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f22bcb826c2357 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.878+0000 7faa96000640 1 -- 172.21.15.94:0/1906528340 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0031db0 con 0x561b60bca5b0 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.878+0000 7faa96000640 1 -- 172.21.15.94:0/1906528340 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0036040 con 0x561b60bca5b0 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.878+0000 7faa96000640 1 -- 172.21.15.94:0/1906528340 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0004360 con 0x561b60bca5b0 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.878+0000 7faaa5600640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faa900a2e50 0x7faa900a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.878+0000 7faaad000640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faa900a2e50 0x7faa900a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.878+0000 7faaa5600640 1 -- 172.21.15.94:0/224066222 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9563+0+0 (secure 0 0 0) 0x7faa9c0cfb90 con 0x7faaa810a430 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 -- 172.21.15.94:0/1906528340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bca5b0 msgr2=0x561b60bc8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bca5b0 0x561b60bc8b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faaa002f530 tx=0x7faaa0005f90 comp rx=0 tx=0).stop 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 -- 172.21.15.94:0/1906528340 shutdown_connections 2024-06-03T01:20:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faaad000640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faa900a2e50 0x7faa900a5310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x561b60bcbd10 tx=0x7faa9c0d3640 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bc94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 --2- 172.21.15.94:0/1906528340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bca5b0 0x561b60bc8b00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 -- 172.21.15.94:0/1906528340 >> 172.21.15.94:0/1906528340 conn(0x561b60b91f80 msgr2=0x561b60bc5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 -- 172.21.15.94:0/1906528340 shutdown_connections 2024-06-03T01:20:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 -- 172.21.15.94:0/1906528340 wait complete. 2024-06-03T01:20:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 -- start start 2024-06-03T01:20:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.879+0000 7faab27fc740 1 Processor -- start 2024-06-03T01:20:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faab27fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bd0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faab27fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bca5b0 0x561b60bd1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faab27fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faab27fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561b60bd69b0 con 0x561b60bca5b0 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faab27fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561b60ae80b0 con 0x561b60bc9040 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faab27fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561b60bd6b30 con 0x561b60bd3dc0 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faab27fc740 10 client.0 _ll_register_callbacks cb 0x561b60ad3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faaa7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bca5b0 0x561b60bd1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faaada00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faaa7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bca5b0 0x561b60bd1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32866/0 (socket says 172.21.15.94:32866) 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faaa7e00640 1 -- 172.21.15.94:0/2905214425 learned_addr learned my addr 172.21.15.94:0/2905214425 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.880+0000 7faaada00640 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bd0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faaa7e00640 1 -- 172.21.15.94:0/2905214425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 msgr2=0x561b60bd18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faaa7e00640 1 -- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 msgr2=0x561b60bd0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bd0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faaa7e00640 1 -- 172.21.15.94:0/2905214425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6090d620 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bca5b0 0x561b60bd1370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa98012d20 tx=0x7faa9800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f844810cd1dac58e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faa8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faa8aa00640 1 -- 172.21.15.94:0/2905214425 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98012110 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faa8aa00640 1 -- 172.21.15.94:0/2905214425 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9563+0+0 (secure 0 0 0) 0x7faa98004050 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa0036040 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faa8aa00640 1 -- 172.21.15.94:0/2905214425 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faa980122b0 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faab27fc740 10 client.15960 fetch_fsmap learned FSMap version 46 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.881+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561b60be2e00 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faa8aa00640 1 -- 172.21.15.94:0/2905214425 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0036040 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faa8aa00640 1 -- 172.21.15.94:0/2905214425 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7faa98010430 con 0x561b60bca5b0 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faab27fc740 10 client.15960 fetch_fsmap finished waiting for FSMap version 46 2024-06-03T01:20:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faab27fc740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faab27fc740 20 client.15960 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faab27fc740 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60be42c0 0x561b60be6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faab27fc740 4 client.15960 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b60b3ab50 con 0x561b60be42c0 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.882+0000 7faaada00640 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60be42c0 0x561b60be6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.883+0000 7faaada00640 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60be42c0 0x561b60be6720 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311bcc8debb680b8 server_cookie=a661a22c0e381c93 in_seq=0 out_seq=0 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.883+0000 7faa8aa00640 10 client.15960 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faa8aa00640 1 -- 172.21.15.94:0/2905214425 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561b60b3ab50 con 0x561b60be42c0 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faa8aa00640 10 client.15960 handle_command_reply: tid=0 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 client.15960 shutdown 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bca5b0 msgr2=0x561b60bd1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bca5b0 0x561b60bd1370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa98012d20 tx=0x7faa9800f790 comp rx=0 tx=0).stop 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 shutdown_connections 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bd3dc0 0x561b60bd18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bca5b0 0x561b60bd1370 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60be42c0 0x561b60be6720 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 --2- 172.21.15.94:0/2905214425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bd0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 >> 172.21.15.94:0/2905214425 conn(0x561b60b91f80 msgr2=0x561b60bcb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faa8aa00640 0 client.15960 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 shutdown_connections 2024-06-03T01:20:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 1 -- 172.21.15.94:0/2905214425 wait complete. 2024-06-03T01:20:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.884+0000 7faab27fc740 20 client.15960 trim_cache size 0 max 16384 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 Processor -- start 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 -- start start 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bcb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bebff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bd3dc0 0x561b60bec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81b9860 con 0x561b60bc9040 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81bb490 con 0x561b60bd3dc0 2024-06-03T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faab27fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81b9f30 con 0x561b60bca5b0 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faaada00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bcb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faaada00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bcb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32874/0 (socket says 172.21.15.94:32874) 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.901+0000 7faaada00640 1 -- 172.21.15.94:0/2965604547 learned_addr learned my addr 172.21.15.94:0/2965604547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bebff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaada00640 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bd3dc0 0x561b60bec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaada00640 1 -- 172.21.15.94:0/2965604547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 msgr2=0x561b60bebff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaada00640 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bebff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaada00640 1 -- 172.21.15.94:0/2965604547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bd3dc0 msgr2=0x561b60bec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaada00640 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bd3dc0 0x561b60bec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaada00640 1 -- 172.21.15.94:0/2965604547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b60bc9650 con 0x561b60bc9040 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bebff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faaada00640 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bcb2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faaa0031a60 tx=0x7faaa002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27118873c3e53742 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faa95600640 1 -- 172.21.15.94:0/2965604547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0031510 con 0x561b60bc9040 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faa95600640 1 -- 172.21.15.94:0/2965604547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0036040 con 0x561b60bc9040 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.902+0000 7faa95600640 1 -- 172.21.15.94:0/2965604547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa00046f0 con 0x561b60bc9040 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 -- 172.21.15.94:0/2965604547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 msgr2=0x561b60bcb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bcb2b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faaa0031a60 tx=0x7faaa002a8c0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 -- 172.21.15.94:0/2965604547 shutdown_connections 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bd3dc0 0x561b60bec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bebff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 --2- 172.21.15.94:0/2965604547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bc9040 0x561b60bcb2b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 -- 172.21.15.94:0/2965604547 >> 172.21.15.94:0/2965604547 conn(0x561b60b91f80 msgr2=0x561b60bcaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 -- 172.21.15.94:0/2965604547 shutdown_connections 2024-06-03T01:20:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.903+0000 7faab27fc740 1 -- 172.21.15.94:0/2965604547 wait complete. 2024-06-03T01:20:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 -- start start 2024-06-03T01:20:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 Processor -- start 2024-06-03T01:20:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bdc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bd7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bd3dc0 0x561b60bd7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81bad60 con 0x561b60bd3dc0 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81ba600 con 0x561b60bc9040 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81b7860 con 0x561b60bca5b0 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.904+0000 7faab27fc740 10 client.0 _ll_register_callbacks cb 0x7faa98009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaada00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bd3dc0 0x561b60bd7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bdc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bd7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaada00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bdc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:43278/0 (socket says 172.21.15.94:43278) 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bd7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53566/0 (socket says 172.21.15.94:53566) 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaada00640 1 -- 172.21.15.94:0/2039579695 learned_addr learned my addr 172.21.15.94:0/2039579695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 -- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 msgr2=0x561b60bdc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bdc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 -- 172.21.15.94:0/2039579695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bd3dc0 msgr2=0x561b60bd7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bd3dc0 0x561b60bd7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 -- 172.21.15.94:0/2039579695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b60bd8000 con 0x561b60bca5b0 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaada00640 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bdc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faaa7e00640 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bd7580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa98010850 tx=0x7faa9800fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce39df38b9f7d3db server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.905+0000 7faa8aa00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.906+0000 7faa8aa00640 1 -- 172.21.15.94:0/2039579695 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98000e60 con 0x561b60bca5b0 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.906+0000 7faa8aa00640 1 -- 172.21.15.94:0/2039579695 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9563+0+0 (secure 0 0 0) 0x7faa980050d0 con 0x561b60bca5b0 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.906+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa0036040 con 0x561b60bca5b0 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.906+0000 7faa8aa00640 1 -- 172.21.15.94:0/2039579695 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faa98038050 con 0x561b60bca5b0 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.906+0000 7faab27fc740 10 client.26089 fetch_fsmap learned FSMap version 46 2024-06-03T01:20:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.906+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561b60c4ec90 con 0x561b60bca5b0 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.906+0000 7faa8aa00640 1 -- 172.21.15.94:0/2039579695 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0036040 con 0x561b60bca5b0 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faa8aa00640 1 -- 172.21.15.94:0/2039579695 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7faa98003ad0 con 0x561b60bca5b0 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faab27fc740 10 client.26089 fetch_fsmap finished waiting for FSMap version 46 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faab27fc740 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faab27fc740 20 client.26089 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faab27fc740 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60c4f410 0x561b60c51830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faab27fc740 4 client.26089 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b60b34ea0 con 0x561b60c4f410 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faaada00640 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60c4f410 0x561b60c51830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faaada00640 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60c4f410 0x561b60c51830 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c39b5aa30e05e3 server_cookie=cfc2b491b0b20c28 in_seq=0 out_seq=0 2024-06-03T01:20:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.907+0000 7faa8aa00640 10 client.26089 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faa8aa00640 1 -- 172.21.15.94:0/2039579695 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561b60b34ea0 con 0x561b60c4f410 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faa8aa00640 10 client.26089 handle_command_reply: tid=0 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 client.26089 shutdown 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 msgr2=0x561b60bd7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bd7580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa98010850 tx=0x7faa9800fc90 comp rx=0 tx=0).stop 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 shutdown_connections 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b60bd3dc0 0x561b60bd7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561b60bca5b0 0x561b60bd7580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561b60c4f410 0x561b60c51830 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 --2- 172.21.15.94:0/2039579695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561b60bc9040 0x561b60bdc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 >> 172.21.15.94:0/2039579695 conn(0x561b60b91f80 msgr2=0x561b60bc4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faa8aa00640 0 client.26089 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 shutdown_connections 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 1 -- 172.21.15.94:0/2039579695 wait complete. 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.932+0000 7faab27fc740 20 client.26089 trim_cache size 0 max 16384 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:20:05.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:20:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:20:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faa900a2e50 msgr2=0x7faa900a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faa900a2e50 0x7faa900a5310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x561b60bcbd10 tx=0x7faa9c0d3640 comp rx=0 tx=0).stop 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810a430 msgr2=0x7faaa806b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810a430 0x7faaa806b410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00e9f0 tx=0x7faa9c00eeb0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 shutdown_connections 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7faa900a2e50 0x7faa900a5310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaa806be90 0x7faaa81aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faaa810f4b0 0x7faaa806b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 --2- 172.21.15.94:0/224066222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faaa810a430 0x7faaa806b410 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.933+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 >> 172.21.15.94:0/224066222 conn(0x7faaa806eb20 msgr2=0x7faaa806f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.934+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 shutdown_connections 2024-06-03T01:20:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:05.934+0000 7faaafa00640 1 -- 172.21.15.94:0/224066222 wait complete. 2024-06-03T01:20:06.121 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220688 2.11 MB/sec execute 373 sec latency 366.807 ms 2024-06-03T01:20:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:06 smithi107 ceph-mon[36622]: pgmap v302: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 890 KiB/s wr, 122 op/s 2024-06-03T01:20:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:06 smithi094 ceph-mon[32119]: pgmap v302: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 890 KiB/s wr, 122 op/s 2024-06-03T01:20:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:06 smithi179 ceph-mon[37041]: pgmap v302: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 890 KiB/s wr, 122 op/s 2024-06-03T01:20:07.121 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220688 2.10 MB/sec execute 374 sec latency 1009.029 ms 2024-06-03T01:20:08.121 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220688 2.10 MB/sec execute 375 sec latency 2009.129 ms 2024-06-03T01:20:08.652 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:08 smithi094 ceph-mon[32119]: pgmap v303: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-03T01:20:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:08 smithi107 ceph-mon[36622]: pgmap v303: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-03T01:20:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:08 smithi179 ceph-mon[37041]: pgmap v303: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-03T01:20:09.121 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220688 2.09 MB/sec execute 376 sec latency 3009.286 ms 2024-06-03T01:20:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:09 smithi107 ceph-mon[36622]: osdmap e109: 12 total, 12 up, 12 in 2024-06-03T01:20:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:09 smithi107 ceph-mon[36622]: pgmap v305: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-03T01:20:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:09 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:09 smithi094 ceph-mon[32119]: osdmap e109: 12 total, 12 up, 12 in 2024-06-03T01:20:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:09 smithi094 ceph-mon[32119]: pgmap v305: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-03T01:20:10.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:09 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:09 smithi179 ceph-mon[37041]: osdmap e109: 12 total, 12 up, 12 in 2024-06-03T01:20:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:09 smithi179 ceph-mon[37041]: pgmap v305: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-03T01:20:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:09 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:10.121 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220688 2.08 MB/sec execute 377 sec latency 4009.428 ms 2024-06-03T01:20:11.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220746 2.08 MB/sec execute 378 sec latency 4942.455 ms 2024-06-03T01:20:12.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220756 2.07 MB/sec execute 379 sec latency 978.290 ms 2024-06-03T01:20:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:12 smithi094 ceph-mon[32119]: pgmap v306: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 841 KiB/s wr, 90 op/s 2024-06-03T01:20:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:12 smithi179 ceph-mon[37041]: pgmap v306: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 841 KiB/s wr, 90 op/s 2024-06-03T01:20:12.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:12 smithi107 ceph-mon[36622]: pgmap v306: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 841 KiB/s wr, 90 op/s 2024-06-03T01:20:13.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220756 2.07 MB/sec execute 380 sec latency 1978.384 ms 2024-06-03T01:20:14.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220756 2.06 MB/sec execute 381 sec latency 2978.533 ms 2024-06-03T01:20:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:14 smithi094 ceph-mon[32119]: pgmap v307: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 855 KiB/s wr, 105 op/s 2024-06-03T01:20:14.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:14 smithi179 ceph-mon[37041]: pgmap v307: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 855 KiB/s wr, 105 op/s 2024-06-03T01:20:14.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:14 smithi107 ceph-mon[36622]: pgmap v307: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 855 KiB/s wr, 105 op/s 2024-06-03T01:20:15.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220756 2.06 MB/sec execute 382 sec latency 3978.640 ms 2024-06-03T01:20:15.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:15 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:15 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:15.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:15 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:16.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 220821 2.05 MB/sec execute 383 sec latency 4911.499 ms 2024-06-03T01:20:16.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:16 smithi107 ceph-mon[36622]: pgmap v308: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 740 KiB/s wr, 91 op/s 2024-06-03T01:20:16.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:16 smithi094 ceph-mon[32119]: pgmap v308: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 740 KiB/s wr, 91 op/s 2024-06-03T01:20:16.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:16 smithi179 ceph-mon[37041]: pgmap v308: 129 pgs: 129 active+clean; 85 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 740 KiB/s wr, 91 op/s 2024-06-03T01:20:17.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 221657 2.06 MB/sec execute 384 sec latency 196.116 ms 2024-06-03T01:20:18.123 INFO:tasks.workunit.client.0.smithi094.stdout: 1 221657 2.05 MB/sec execute 385 sec latency 1196.271 ms 2024-06-03T01:20:18.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:18 smithi094 ceph-mon[32119]: pgmap v309: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 468 KiB/s wr, 48 op/s 2024-06-03T01:20:18.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:18 smithi179 ceph-mon[37041]: pgmap v309: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 468 KiB/s wr, 48 op/s 2024-06-03T01:20:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:18 smithi107 ceph-mon[36622]: pgmap v309: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 468 KiB/s wr, 48 op/s 2024-06-03T01:20:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 -- 172.21.15.94:0/3716451751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 msgr2=0x7f3864107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 --2- 172.21.15.94:0/3716451751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 0x7f3864107d70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3854009ca0 tx=0x7f385402a8c0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 -- 172.21.15.94:0/3716451751 shutdown_connections 2024-06-03T01:20:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 --2- 172.21.15.94:0/3716451751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864102340 0x7f386410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 --2- 172.21.15.94:0/3716451751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3864101980 0x7f3864101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 --2- 172.21.15.94:0/3716451751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 0x7f3864107d70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 -- 172.21.15.94:0/3716451751 >> 172.21.15.94:0/3716451751 conn(0x7f38640fd450 msgr2=0x7f38640ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 -- 172.21.15.94:0/3716451751 shutdown_connections 2024-06-03T01:20:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.075+0000 7f386ac00640 1 -- 172.21.15.94:0/3716451751 wait complete. 2024-06-03T01:20:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.076+0000 7f386ac00640 1 Processor -- start 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.076+0000 7f386ac00640 1 -- start start 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f386ac00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864101980 0x7f38641981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f386ac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3864102340 0x7f3864198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f386ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 0x7f386419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f386ac00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3864111df0 con 0x7f3864101980 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f386ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3864111c70 con 0x7f3864107990 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f386ac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3864111f70 con 0x7f3864102340 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f3863e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864101980 0x7f38641981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f3863e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864101980 0x7f38641981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34306/0 (socket says 172.21.15.94:34306) 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f3863e00640 1 -- 172.21.15.94:0/3536830940 learned_addr learned my addr 172.21.15.94:0/3536830940 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f3868c00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 0x7f386419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.077+0000 7f3863400640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3864102340 0x7f3864198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3863e00640 1 -- 172.21.15.94:0/3536830940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3864102340 msgr2=0x7f3864198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3863e00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3864102340 0x7f3864198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3863e00640 1 -- 172.21.15.94:0/3536830940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 msgr2=0x7f386419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3863e00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 0x7f386419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3863e00640 1 -- 172.21.15.94:0/3536830940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386419d1c0 con 0x7f3864101980 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3863e00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864101980 0x7f38641981d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3854005e30 tx=0x7f3854003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3860c00640 1 -- 172.21.15.94:0/3536830940 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854031650 con 0x7f3864101980 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3860c00640 1 -- 172.21.15.94:0/3536830940 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38540317f0 con 0x7f3864101980 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386419d450 con 0x7f3864101980 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f3860c00640 1 -- 172.21.15.94:0/3536830940 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385402faa0 con 0x7f3864101980 2024-06-03T01:20:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.078+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38641a4d40 con 0x7f3864101980 2024-06-03T01:20:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.082+0000 7f3860c00640 1 -- 172.21.15.94:0/3536830940 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f385402fc40 con 0x7f3864101980 2024-06-03T01:20:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.082+0000 7f386da10740 1 Processor -- start 2024-06-03T01:20:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.082+0000 7f386da10740 1 -- start start 2024-06-03T01:20:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f386da10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2748b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f386da10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e27494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f386da10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e27561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f386da10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5587e26ad420 con 0x5587e2749040 2024-06-03T01:20:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f386da10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5587e2654c50 con 0x5587e2753dc0 2024-06-03T01:20:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f386da10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5587e26a93e0 con 0x5587e274a5b0 2024-06-03T01:20:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3863e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e27494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3863e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e27494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34312/0 (socket says 172.21.15.94:34312) 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3868c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2748b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3863e00640 1 -- 172.21.15.94:0/2121846276 learned_addr learned my addr 172.21.15.94:0/2121846276 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3863400640 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e27561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3868c00640 1 -- 172.21.15.94:0/2121846276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 msgr2=0x5587e27561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3868c00640 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e27561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3868c00640 1 -- 172.21.15.94:0/2121846276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 msgr2=0x5587e27494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3868c00640 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e27494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.083+0000 7f3868c00640 1 -- 172.21.15.94:0/2121846276 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e23c1210 con 0x5587e274a5b0 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.084+0000 7f3860c00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38480a2da0 0x7f38480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.084+0000 7f3868c00640 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2748b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f385c009bb0 tx=0x7f385c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49dd52459b3be8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.084+0000 7f3860c00640 1 -- 172.21.15.94:0/3536830940 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f38540f0a70 con 0x7f3864101980 2024-06-03T01:20:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.084+0000 7f3863e00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38480a2da0 0x7f38480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.084+0000 7f3859200640 1 -- 172.21.15.94:0/2121846276 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c004050 con 0x5587e274a5b0 2024-06-03T01:20:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.084+0000 7f3859200640 1 -- 172.21.15.94:0/2121846276 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385c005500 con 0x5587e274a5b0 2024-06-03T01:20:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.084+0000 7f3859200640 1 -- 172.21.15.94:0/2121846276 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c004cd0 con 0x5587e274a5b0 2024-06-03T01:20:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 -- 172.21.15.94:0/2121846276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 msgr2=0x5587e2748b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2748b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f385c009bb0 tx=0x7f385c011810 comp rx=0 tx=0).stop 2024-06-03T01:20:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f3863e00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38480a2da0 0x7f38480a5260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38540bcc10 tx=0x7f38540f44a0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 -- 172.21.15.94:0/2121846276 shutdown_connections 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e27561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e27494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 --2- 172.21.15.94:0/2121846276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2748b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 -- 172.21.15.94:0/2121846276 >> 172.21.15.94:0/2121846276 conn(0x5587e2711f80 msgr2=0x5587e2745630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 -- 172.21.15.94:0/2121846276 shutdown_connections 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.085+0000 7f386da10740 1 -- 172.21.15.94:0/2121846276 wait complete. 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 -- start start 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 Processor -- start 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2749040 0x5587e2750e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e274a5b0 0x5587e2751370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e2753dc0 0x5587e27518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5587e2756b90 con 0x5587e274a5b0 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5587e26680b0 con 0x5587e2749040 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5587e2756d10 con 0x5587e2753dc0 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.086+0000 7f386da10740 10 client.0 _ll_register_callbacks cb 0x5587e2653f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3868c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e2753dc0 0x5587e27518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3868c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e2753dc0 0x5587e27518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43252/0 (socket says 172.21.15.94:43252) 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3868c00640 1 -- 172.21.15.94:0/3203357266 learned_addr learned my addr 172.21.15.94:0/3203357266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3863400640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e274a5b0 0x5587e2751370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3868c00640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2749040 0x5587e2750e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3863400640 1 -- 172.21.15.94:0/3203357266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e2753dc0 msgr2=0x5587e27518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3863400640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e2753dc0 0x5587e27518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3863400640 1 -- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2749040 msgr2=0x5587e2750e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3863400640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2749040 0x5587e2750e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3863400640 1 -- 172.21.15.94:0/3203357266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e248d620 con 0x5587e274a5b0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3868c00640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e2753dc0 0x5587e27518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3863400640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e274a5b0 0x5587e2751370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5587e2749ae0 tx=0x7f384c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8eac71b97efc88 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.087+0000 7f3851600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.088+0000 7f3851600640 1 -- 172.21.15.94:0/3203357266 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c004020 con 0x5587e274a5b0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.088+0000 7f3851600640 1 -- 172.21.15.94:0/3203357266 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f384c0041c0 con 0x5587e274a5b0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.088+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f385c005500 con 0x5587e274a5b0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.088+0000 7f3851600640 1 -- 172.21.15.94:0/3203357266 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f384c025050 con 0x5587e274a5b0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.088+0000 7f386da10740 10 client.15978 fetch_fsmap learned FSMap version 48 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.088+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5587e2762fe0 con 0x5587e274a5b0 2024-06-03T01:20:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.088+0000 7f3851600640 1 -- 172.21.15.94:0/3203357266 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385c005500 con 0x5587e274a5b0 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f3851600640 1 -- 172.21.15.94:0/3203357266 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f384c003a30 con 0x5587e274a5b0 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f386da10740 10 client.15978 fetch_fsmap finished waiting for FSMap version 48 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f386da10740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f386da10740 20 client.15978 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f386da10740 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27644a0 0x5587e2766890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f386da10740 4 client.15978 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e26bab50 con 0x5587e27644a0 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.089+0000 7f3868c00640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27644a0 0x5587e2766890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.090+0000 7f3868c00640 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27644a0 0x5587e2766890 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1443bed1063a71da server_cookie=d1f1d9e9f26f237b in_seq=0 out_seq=0 2024-06-03T01:20:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.090+0000 7f3851600640 10 client.15978 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.090+0000 7f3851600640 1 -- 172.21.15.94:0/3203357266 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5587e26bab50 con 0x5587e27644a0 2024-06-03T01:20:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.090+0000 7f3851600640 10 client.15978 handle_command_reply: tid=0 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.090+0000 7f386da10740 1 client.15978 shutdown 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e274a5b0 msgr2=0x5587e2751370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e274a5b0 0x5587e2751370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5587e2749ae0 tx=0x7f384c012b60 comp rx=0 tx=0).stop 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 shutdown_connections 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e2753dc0 0x5587e27518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e274a5b0 0x5587e2751370 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27644a0 0x5587e2766890 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 --2- 172.21.15.94:0/3203357266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2749040 0x5587e2750e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 >> 172.21.15.94:0/3203357266 conn(0x5587e2711f80 msgr2=0x5587e274b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f3851600640 0 client.15978 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 shutdown_connections 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 1 -- 172.21.15.94:0/3203357266 wait complete. 2024-06-03T01:20:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.091+0000 7f386da10740 20 client.15978 trim_cache size 0 max 16384 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.107+0000 7f386da10740 1 Processor -- start 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f386da10740 1 -- start start 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f386da10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e2744690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f386da10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e276c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f386da10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e276c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f386da10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38641aee10 con 0x5587e2749040 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f386da10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38641b09e0 con 0x5587e2753dc0 2024-06-03T01:20:19.108 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f386da10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38641af4e0 con 0x5587e274a5b0 2024-06-03T01:20:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f3868c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e2744690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f3863400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e276c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f3868c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e2744690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34316/0 (socket says 172.21.15.94:34316) 2024-06-03T01:20:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.108+0000 7f3868c00640 1 -- 172.21.15.94:0/1701866839 learned_addr learned my addr 172.21.15.94:0/1701866839 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:19.110 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3868c00640 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e276c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3868c00640 1 -- 172.21.15.94:0/1701866839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 msgr2=0x5587e276c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3868c00640 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e276c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3868c00640 1 -- 172.21.15.94:0/1701866839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 msgr2=0x5587e276c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3868c00640 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e276c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3868c00640 1 -- 172.21.15.94:0/1701866839 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e274f380 con 0x5587e2749040 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3863400640 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e276c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3868c00640 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e2744690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f385c014040 tx=0x7f385c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc462326f07ff19 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3853e00640 1 -- 172.21.15.94:0/1701866839 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c004250 con 0x5587e2749040 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3853e00640 1 -- 172.21.15.94:0/1701866839 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385c02c030 con 0x5587e2749040 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.109+0000 7f3853e00640 1 -- 172.21.15.94:0/1701866839 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c003930 con 0x5587e2749040 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 -- 172.21.15.94:0/1701866839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 msgr2=0x5587e2744690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e2744690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f385c014040 tx=0x7f385c000bc0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 -- 172.21.15.94:0/1701866839 shutdown_connections 2024-06-03T01:20:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e276c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e276c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 --2- 172.21.15.94:0/1701866839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e2744690 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 -- 172.21.15.94:0/1701866839 >> 172.21.15.94:0/1701866839 conn(0x5587e2711f80 msgr2=0x5587e2743f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 -- 172.21.15.94:0/1701866839 shutdown_connections 2024-06-03T01:20:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.110+0000 7f386da10740 1 -- 172.21.15.94:0/1701866839 wait complete. 2024-06-03T01:20:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 -- start start 2024-06-03T01:20:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 Processor -- start 2024-06-03T01:20:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e275c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2757a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e2757fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38641b0310 con 0x5587e2749040 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38641afbb0 con 0x5587e2753dc0 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38641ace10 con 0x5587e274a5b0 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.111+0000 7f386da10740 10 client.0 _ll_register_callbacks cb 0x7f384c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3868c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e275c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3868c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e275c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34322/0 (socket says 172.21.15.94:34322) 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3868c00640 1 -- 172.21.15.94:0/4185062892 learned_addr learned my addr 172.21.15.94:0/4185062892 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3863400640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2757a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3868c00640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e2757fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3863400640 1 -- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 msgr2=0x5587e2757fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3863400640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e2757fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3863400640 1 -- 172.21.15.94:0/4185062892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 msgr2=0x5587e275c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3863400640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e275c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3863400640 1 -- 172.21.15.94:0/4185062892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e2758800 con 0x5587e274a5b0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3868c00640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e275c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3863400640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2757a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f384c0059f0 tx=0x7f384c005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c32e302d0e044cb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.112+0000 7f3851600640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f3851600640 1 -- 172.21.15.94:0/4185062892 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c02e060 con 0x5587e274a5b0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f385c02c030 con 0x5587e274a5b0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f3851600640 1 -- 172.21.15.94:0/4185062892 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f384c018070 con 0x5587e274a5b0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f3851600640 1 -- 172.21.15.94:0/4185062892 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f384c035050 con 0x5587e274a5b0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f386da10740 10 client.26107 fetch_fsmap learned FSMap version 48 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5587e27cf130 con 0x5587e274a5b0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f3851600640 1 -- 172.21.15.94:0/4185062892 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385c02c030 con 0x5587e274a5b0 2024-06-03T01:20:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.113+0000 7f3851600640 1 -- 172.21.15.94:0/4185062892 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f384c0035f0 con 0x5587e274a5b0 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.114+0000 7f386da10740 10 client.26107 fetch_fsmap finished waiting for FSMap version 48 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.114+0000 7f386da10740 10 client.26107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.114+0000 7f386da10740 20 client.26107 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.114+0000 7f386da10740 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27cf850 0x5587e27d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.114+0000 7f386da10740 4 client.26107 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.114+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587e26b4ea0 con 0x5587e27cf850 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.114+0000 7f3868c00640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27cf850 0x5587e27d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.115+0000 7f3851600640 10 client.26107 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.115+0000 7f3868c00640 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27cf850 0x5587e27d1c70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16380f211bf29afb server_cookie=fce9f64a54ad2699 in_seq=0 out_seq=0 2024-06-03T01:20:19.122 INFO:tasks.workunit.client.0.smithi094.stdout: 1 221657 2.05 MB/sec execute 386 sec latency 2196.361 ms 2024-06-03T01:20:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f3851600640 1 -- 172.21.15.94:0/4185062892 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5587e26b4ea0 con 0x5587e27cf850 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f3851600640 10 client.26107 handle_command_reply: tid=0 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 client.26107 shutdown 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 msgr2=0x5587e2757a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2757a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f384c0059f0 tx=0x7f384c005d40 comp rx=0 tx=0).stop 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 shutdown_connections 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5587e27cf850 0x5587e27d1c70 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5587e2753dc0 0x5587e2757fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5587e274a5b0 0x5587e2757a90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f3851600640 0 client.26107 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 --2- 172.21.15.94:0/4185062892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5587e2749040 0x5587e275c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.137+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 >> 172.21.15.94:0/4185062892 conn(0x5587e2711f80 msgr2=0x5587e274b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.138+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 shutdown_connections 2024-06-03T01:20:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.138+0000 7f386da10740 1 -- 172.21.15.94:0/4185062892 wait complete. 2024-06-03T01:20:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.138+0000 7f386da10740 20 client.26107 trim_cache size 0 max 16384 2024-06-03T01:20:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:20:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:20:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:20:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38480a2da0 msgr2=0x7f38480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38480a2da0 0x7f38480a5260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f38540bcc10 tx=0x7f38540f44a0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864101980 msgr2=0x7f38641981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864101980 0x7f38641981d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3854005e30 tx=0x7f3854003f90 comp rx=0 tx=0).stop 2024-06-03T01:20:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 shutdown_connections 2024-06-03T01:20:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38480a2da0 0x7f38480a5260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3864107990 0x7f386419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3864102340 0x7f3864198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 --2- 172.21.15.94:0/3536830940 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3864101980 0x7f38641981d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 >> 172.21.15.94:0/3536830940 conn(0x7f38640fd450 msgr2=0x7f38640fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 shutdown_connections 2024-06-03T01:20:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:19.139+0000 7f386ac00640 1 -- 172.21.15.94:0/3536830940 wait complete. 2024-06-03T01:20:20.123 INFO:tasks.workunit.client.0.smithi094.stdout: 1 221657 2.04 MB/sec execute 387 sec latency 3196.489 ms 2024-06-03T01:20:20.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:20 smithi179 ceph-mon[37041]: pgmap v310: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 453 KiB/s wr, 46 op/s 2024-06-03T01:20:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:20 smithi107 ceph-mon[36622]: pgmap v310: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 453 KiB/s wr, 46 op/s 2024-06-03T01:20:20.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:20 smithi094 ceph-mon[32119]: pgmap v310: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 453 KiB/s wr, 46 op/s 2024-06-03T01:20:21.123 INFO:tasks.workunit.client.0.smithi094.stdout: 1 222742 2.05 MB/sec execute 388 sec latency 3393.526 ms 2024-06-03T01:20:22.123 INFO:tasks.workunit.client.0.smithi094.stdout: 1 223809 2.07 MB/sec execute 389 sec latency 17.446 ms 2024-06-03T01:20:22.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:22 smithi107 ceph-mon[36622]: pgmap v311: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 390 KiB/s wr, 40 op/s 2024-06-03T01:20:22.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:22 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:22.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:22 smithi094 ceph-mon[32119]: pgmap v311: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 390 KiB/s wr, 40 op/s 2024-06-03T01:20:22.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:22 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:22 smithi179 ceph-mon[37041]: pgmap v311: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 390 KiB/s wr, 40 op/s 2024-06-03T01:20:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:22 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:23.123 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224236 2.06 MB/sec execute 390 sec latency 258.914 ms 2024-06-03T01:20:24.123 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224236 2.06 MB/sec execute 391 sec latency 1259.094 ms 2024-06-03T01:20:24.407 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:24 smithi179 ceph-mon[37041]: pgmap v312: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 544 KiB/s wr, 69 op/s 2024-06-03T01:20:24.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:24 smithi107 ceph-mon[36622]: pgmap v312: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 544 KiB/s wr, 69 op/s 2024-06-03T01:20:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:24 smithi094 ceph-mon[32119]: pgmap v312: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 544 KiB/s wr, 69 op/s 2024-06-03T01:20:25.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224236 2.05 MB/sec execute 392 sec latency 2259.266 ms 2024-06-03T01:20:26.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224303 2.05 MB/sec execute 393 sec latency 3192.108 ms 2024-06-03T01:20:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:26 smithi107 ceph-mon[36622]: pgmap v313: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 489 KiB/s wr, 54 op/s 2024-06-03T01:20:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:26 smithi094 ceph-mon[32119]: pgmap v313: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 489 KiB/s wr, 54 op/s 2024-06-03T01:20:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:26 smithi179 ceph-mon[37041]: pgmap v313: 129 pgs: 129 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 489 KiB/s wr, 54 op/s 2024-06-03T01:20:27.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224304 2.04 MB/sec execute 394 sec latency 993.210 ms 2024-06-03T01:20:28.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224304 2.04 MB/sec execute 395 sec latency 1993.298 ms 2024-06-03T01:20:28.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:28 smithi107 ceph-mon[36622]: pgmap v314: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 620 KiB/s wr, 80 op/s 2024-06-03T01:20:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:28 smithi094 ceph-mon[32119]: pgmap v314: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 620 KiB/s wr, 80 op/s 2024-06-03T01:20:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:28 smithi179 ceph-mon[37041]: pgmap v314: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 620 KiB/s wr, 80 op/s 2024-06-03T01:20:29.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224304 2.03 MB/sec execute 396 sec latency 2993.444 ms 2024-06-03T01:20:30.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224304 2.03 MB/sec execute 397 sec latency 3993.595 ms 2024-06-03T01:20:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:30 smithi107 ceph-mon[36622]: pgmap v315: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 285 KiB/s wr, 55 op/s 2024-06-03T01:20:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:30 smithi094 ceph-mon[32119]: pgmap v315: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 285 KiB/s wr, 55 op/s 2024-06-03T01:20:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:30 smithi179 ceph-mon[37041]: pgmap v315: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 285 KiB/s wr, 55 op/s 2024-06-03T01:20:31.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 224347 2.02 MB/sec execute 398 sec latency 4927.080 ms 2024-06-03T01:20:32.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 225205 2.03 MB/sec execute 399 sec latency 173.917 ms 2024-06-03T01:20:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 -- 172.21.15.94:0/754982657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 msgr2=0x7f7a5810b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 --2- 172.21.15.94:0/754982657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 0x7f7a5810b820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009900 tx=0x7f7a4402a490 comp rx=0 tx=0).stop 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 -- 172.21.15.94:0/754982657 shutdown_connections 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 --2- 172.21.15.94:0/754982657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5810bdf0 0x7f7a5810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 --2- 172.21.15.94:0/754982657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 0x7f7a5810b820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 --2- 172.21.15.94:0/754982657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 0x7f7a58102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 -- 172.21.15.94:0/754982657 >> 172.21.15.94:0/754982657 conn(0x7f7a580fc1c0 msgr2=0x7f7a580fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 -- 172.21.15.94:0/754982657 shutdown_connections 2024-06-03T01:20:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.279+0000 7f7a5de00640 1 -- 172.21.15.94:0/754982657 wait complete. 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.280+0000 7f7a5de00640 1 Processor -- start 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.280+0000 7f7a5de00640 1 -- start start 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 0x7f7a5819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 0x7f7a5819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5810bdf0 0x7f7a5819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5de00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58111b80 con 0x7f7a58102a20 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58111a00 con 0x7f7a58103340 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58111d00 con 0x7f7a5810bdf0 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 0x7f7a5819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 0x7f7a5819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56696/0 (socket says 172.21.15.94:56696) 2024-06-03T01:20:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 -- 172.21.15.94:0/2085803781 learned_addr learned my addr 172.21.15.94:0/2085803781 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a57e00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 0x7f7a5819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5d400640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5810bdf0 0x7f7a5819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 -- 172.21.15.94:0/2085803781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5810bdf0 msgr2=0x7f7a5819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5810bdf0 0x7f7a5819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 -- 172.21.15.94:0/2085803781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 msgr2=0x7f7a5819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 0x7f7a5819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.281+0000 7f7a5ca00640 1 -- 172.21.15.94:0/2085803781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5819f2f0 con 0x7f7a58102a20 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.282+0000 7f7a5ca00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 0x7f7a5819a300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800ebc0 tx=0x7f7a4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.282+0000 7f7a55600640 1 -- 172.21.15.94:0/2085803781 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48010630 con 0x7f7a58102a20 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.282+0000 7f7a55600640 1 -- 172.21.15.94:0/2085803781 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a48010cb0 con 0x7f7a58102a20 2024-06-03T01:20:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.282+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5819f5e0 con 0x7f7a58102a20 2024-06-03T01:20:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.282+0000 7f7a55600640 1 -- 172.21.15.94:0/2085803781 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48018bd0 con 0x7f7a58102a20 2024-06-03T01:20:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.282+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a581a6e70 con 0x7f7a58102a20 2024-06-03T01:20:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.285+0000 7f7a6236d740 1 Processor -- start 2024-06-03T01:20:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.285+0000 7f7a55600640 1 -- 172.21.15.94:0/2085803781 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7a480107d0 con 0x7f7a58102a20 2024-06-03T01:20:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.285+0000 7f7a6236d740 1 -- start start 2024-06-03T01:20:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb23c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb249370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a6236d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5591bb1a0420 con 0x5591bb23d770 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a6236d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5591bb147c50 con 0x5591bb246f80 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a6236d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5591bb19c3e0 con 0x5591bb23c200 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a5d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a5ca00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb23c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a5d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56710/0 (socket says 172.21.15.94:56710) 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a5d400640 1 -- 172.21.15.94:0/2417459688 learned_addr learned my addr 172.21.15.94:0/2417459688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a57e00640 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb249370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a5d400640 1 -- 172.21.15.94:0/2417459688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 msgr2=0x5591bb23c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.286+0000 7f7a5d400640 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb23c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a5d400640 1 -- 172.21.15.94:0/2417459688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 msgr2=0x5591bb249370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a5d400640 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb249370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a5d400640 1 -- 172.21.15.94:0/2417459688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591baeb4210 con 0x5591bb23d770 2024-06-03T01:20:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a5d400640 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23bcc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a500045e0 tx=0x7f7a50004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5369704913895a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a4dc00640 1 -- 172.21.15.94:0/2417459688 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50003af0 con 0x5591bb23d770 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a4dc00640 1 -- 172.21.15.94:0/2417459688 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a50015070 con 0x5591bb23d770 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a4dc00640 1 -- 172.21.15.94:0/2417459688 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500116a0 con 0x5591bb23d770 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.287+0000 7f7a55600640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a3c0a2cf0 0x7f7a3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a55600640 1 -- 172.21.15.94:0/2085803781 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f7a480cfe90 con 0x7f7a58102a20 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a5ca00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a3c0a2cf0 0x7f7a3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 -- 172.21.15.94:0/2417459688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 msgr2=0x5591bb23bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23bcc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a500045e0 tx=0x7f7a50004610 comp rx=0 tx=0).stop 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 -- 172.21.15.94:0/2417459688 shutdown_connections 2024-06-03T01:20:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb249370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb23c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2417459688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23bcc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 -- 172.21.15.94:0/2417459688 >> 172.21.15.94:0/2417459688 conn(0x5591bb204f80 msgr2=0x5591bb2387f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 -- 172.21.15.94:0/2417459688 shutdown_connections 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a5ca00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a3c0a2cf0 0x7f7a3c0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5591bb23eed0 tx=0x7f7a480d3940 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.288+0000 7f7a6236d740 1 -- 172.21.15.94:0/2417459688 wait complete. 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 -- start start 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 Processor -- start 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb243f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb244490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb2449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5591bb249d50 con 0x5591bb23d770 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5591bb15b0b0 con 0x5591bb246f80 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.289+0000 7f7a6236d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5591bb249ed0 con 0x5591bb23c200 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a6236d740 10 client.0 _ll_register_callbacks cb 0x5591bb146f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb244490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb244490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56722/0 (socket says 172.21.15.94:56722) 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 -- 172.21.15.94:0/2529582278 learned_addr learned my addr 172.21.15.94:0/2529582278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a5d400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb243f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a5d400640 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb2449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 -- 172.21.15.94:0/2529582278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 msgr2=0x5591bb243f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb243f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 -- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 msgr2=0x5591bb2449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb2449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.290+0000 7f7a57e00640 1 -- 172.21.15.94:0/2529582278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591baf80620 con 0x5591bb23d770 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a57e00640 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb244490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a4402a9b0 tx=0x7f7a44031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7409c2fe899c5856 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a42000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a42000640 1 -- 172.21.15.94:0/2529582278 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44035040 con 0x5591bb23d770 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a42000640 1 -- 172.21.15.94:0/2529582278 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f7a44004240 con 0x5591bb23d770 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a50015070 con 0x5591bb23d770 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a42000640 1 -- 172.21.15.94:0/2529582278 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7a4404f050 con 0x5591bb23d770 2024-06-03T01:20:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a6236d740 10 client.15996 fetch_fsmap learned FSMap version 49 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5591bb256080 con 0x5591bb23d770 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.291+0000 7f7a42000640 1 -- 172.21.15.94:0/2529582278 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a50015070 con 0x5591bb23d770 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a42000640 1 -- 172.21.15.94:0/2529582278 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f7a44046510 con 0x5591bb23d770 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a6236d740 10 client.15996 fetch_fsmap finished waiting for FSMap version 49 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a6236d740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a6236d740 20 client.15996 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb257590 0x5591bb259980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a6236d740 4 client.15996 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591bb1adb50 con 0x5591bb257590 2024-06-03T01:20:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.292+0000 7f7a5d400640 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb257590 0x5591bb259980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.293+0000 7f7a5d400640 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb257590 0x5591bb259980 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40a3c88c7f36710 server_cookie=a619ee8dc89414f8 in_seq=0 out_seq=0 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.293+0000 7f7a42000640 10 client.15996 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a42000640 1 -- 172.21.15.94:0/2529582278 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5591bb1adb50 con 0x5591bb257590 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a42000640 10 client.15996 handle_command_reply: tid=0 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 client.15996 shutdown 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 msgr2=0x5591bb244490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb244490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a4402a9b0 tx=0x7f7a44031850 comp rx=0 tx=0).stop 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 shutdown_connections 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb257590 0x5591bb259980 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb2449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb244490 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 --2- 172.21.15.94:0/2529582278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb243f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a42000640 0 client.15996 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 >> 172.21.15.94:0/2529582278 conn(0x5591bb204f80 msgr2=0x5591bb23e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 shutdown_connections 2024-06-03T01:20:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 1 -- 172.21.15.94:0/2529582278 wait complete. 2024-06-03T01:20:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.294+0000 7f7a6236d740 20 client.15996 trim_cache size 0 max 16384 2024-06-03T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 Processor -- start 2024-06-03T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 -- start start 2024-06-03T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb2447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb25f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581b0f70 con 0x5591bb23d770 2024-06-03T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581b2ae0 con 0x5591bb246f80 2024-06-03T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.311+0000 7f7a6236d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581b1640 con 0x5591bb23c200 2024-06-03T01:20:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56734/0 (socket says 172.21.15.94:56734) 2024-06-03T01:20:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 -- 172.21.15.94:0/1306876091 learned_addr learned my addr 172.21.15.94:0/1306876091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a5d400640 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb25f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a5d400640 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb2447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 -- 172.21.15.94:0/1306876091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 msgr2=0x5591bb2447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb2447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 -- 172.21.15.94:0/1306876091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 msgr2=0x5591bb25f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb25f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a57e00640 1 -- 172.21.15.94:0/1306876091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591bb238d30 con 0x5591bb23d770 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.312+0000 7f7a5d400640 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb25f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.313+0000 7f7a57e00640 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23e470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009850 tx=0x7f7a4402a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce35b582976ba33b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.313+0000 7f7a4d200640 1 -- 172.21.15.94:0/1306876091 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44058070 con 0x5591bb23d770 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.313+0000 7f7a4d200640 1 -- 172.21.15.94:0/1306876091 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a44046680 con 0x5591bb23d770 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.313+0000 7f7a4d200640 1 -- 172.21.15.94:0/1306876091 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44035030 con 0x5591bb23d770 2024-06-03T01:20:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.313+0000 7f7a6236d740 1 -- 172.21.15.94:0/1306876091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 msgr2=0x5591bb23e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.313+0000 7f7a6236d740 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23e470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009850 tx=0x7f7a4402a490 comp rx=0 tx=0).stop 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 -- 172.21.15.94:0/1306876091 shutdown_connections 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb246f80 0x5591bb25f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23d770 0x5591bb23e470 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 --2- 172.21.15.94:0/1306876091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb23c200 0x5591bb2447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 -- 172.21.15.94:0/1306876091 >> 172.21.15.94:0/1306876091 conn(0x5591bb204f80 msgr2=0x5591bb23dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 -- 172.21.15.94:0/1306876091 shutdown_connections 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 -- 172.21.15.94:0/1306876091 wait complete. 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 -- start start 2024-06-03T01:20:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.314+0000 7f7a6236d740 1 Processor -- start 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23c200 0x5591bb24a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb23d770 0x5591bb24ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a6236d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb246f80 0x5591bb2c0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a6236d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581b2410 con 0x5591bb23c200 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a6236d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581b1d10 con 0x5591bb23d770 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a6236d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581aef70 con 0x5591bb246f80 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a6236d740 10 client.0 _ll_register_callbacks cb 0x7f7a44005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a5d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23c200 0x5591bb24a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb23d770 0x5591bb24ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a5d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23c200 0x5591bb24a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56742/0 (socket says 172.21.15.94:56742) 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a5d400640 1 -- 172.21.15.94:0/591435545 learned_addr learned my addr 172.21.15.94:0/591435545 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a5d400640 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb246f80 0x5591bb2c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a57e00640 1 -- 172.21.15.94:0/591435545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb246f80 msgr2=0x5591bb2c0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a57e00640 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb246f80 0x5591bb2c0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a57e00640 1 -- 172.21.15.94:0/591435545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23c200 msgr2=0x5591bb24a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a57e00640 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23c200 0x5591bb24a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a57e00640 1 -- 172.21.15.94:0/591435545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591bb2c0810 con 0x5591bb23d770 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.315+0000 7f7a5d400640 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23c200 0x5591bb24a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a57e00640 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb23d770 0x5591bb24ac80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5591bae85c90 tx=0x7f7a4403d680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16deff8b393a99f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a42000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a42000640 1 -- 172.21.15.94:0/591435545 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44061060 con 0x5591bb23d770 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a44046680 con 0x5591bb23d770 2024-06-03T01:20:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a42000640 1 -- 172.21.15.94:0/591435545 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f7a44005700 con 0x5591bb23d770 2024-06-03T01:20:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a42000640 1 -- 172.21.15.94:0/591435545 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7a44068050 con 0x5591bb23d770 2024-06-03T01:20:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a6236d740 10 client.26114 fetch_fsmap learned FSMap version 49 2024-06-03T01:20:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5591bb2c1b20 con 0x5591bb23d770 2024-06-03T01:20:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.316+0000 7f7a42000640 1 -- 172.21.15.94:0/591435545 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a44046680 con 0x5591bb23d770 2024-06-03T01:20:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a42000640 1 -- 172.21.15.94:0/591435545 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f7a44053530 con 0x5591bb23d770 2024-06-03T01:20:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a6236d740 10 client.26114 fetch_fsmap finished waiting for FSMap version 49 2024-06-03T01:20:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a6236d740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a6236d740 20 client.26114 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a6236d740 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb2c2200 0x5591bb2c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a6236d740 4 client.26114 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:20:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591bb1a7ea0 con 0x5591bb2c2200 2024-06-03T01:20:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a5d400640 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb2c2200 0x5591bb2c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a5d400640 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb2c2200 0x5591bb2c4620 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6d25ea45f77a57 server_cookie=8fc9479e8fd72e59 in_seq=0 out_seq=0 2024-06-03T01:20:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.317+0000 7f7a42000640 10 client.26114 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a42000640 1 -- 172.21.15.94:0/591435545 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5591bb1a7ea0 con 0x5591bb2c2200 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a42000640 10 client.26114 handle_command_reply: tid=0 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 client.26114 shutdown 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb23d770 msgr2=0x5591bb24ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb23d770 0x5591bb24ac80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5591bae85c90 tx=0x7f7a4403d680 comp rx=0 tx=0).stop 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 shutdown_connections 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591bb246f80 0x5591bb2c0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5591bb2c2200 0x5591bb2c4620 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591bb23d770 0x5591bb24ac80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 --2- 172.21.15.94:0/591435545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591bb23c200 0x5591bb24a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a42000640 0 client.26114 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 >> 172.21.15.94:0/591435545 conn(0x5591bb204f80 msgr2=0x5591bb237970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 shutdown_connections 2024-06-03T01:20:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 1 -- 172.21.15.94:0/591435545 wait complete. 2024-06-03T01:20:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.341+0000 7f7a6236d740 20 client.26114 trim_cache size 0 max 16384 2024-06-03T01:20:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:20:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:20:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:20:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.342+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a3c0a2cf0 msgr2=0x7f7a3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.342+0000 7f7a5de00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a3c0a2cf0 0x7f7a3c0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5591bb23eed0 tx=0x7f7a480d3940 comp rx=0 tx=0).stop 2024-06-03T01:20:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.342+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 msgr2=0x7f7a5819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.342+0000 7f7a5de00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 0x7f7a5819a300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800ebc0 tx=0x7f7a4800c6a0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 shutdown_connections 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7a3c0a2cf0 0x7f7a3c0a51b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a5810bdf0 0x7f7a5819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a58103340 0x7f7a5819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 --2- 172.21.15.94:0/2085803781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7a58102a20 0x7f7a5819a300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 >> 172.21.15.94:0/2085803781 conn(0x7f7a580fc1c0 msgr2=0x7f7a581016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 shutdown_connections 2024-06-03T01:20:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:32.343+0000 7f7a5de00640 1 -- 172.21.15.94:0/2085803781 wait complete. 2024-06-03T01:20:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:32 smithi107 ceph-mon[36622]: pgmap v316: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 285 KiB/s wr, 55 op/s 2024-06-03T01:20:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:32 smithi094 ceph-mon[32119]: pgmap v316: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 285 KiB/s wr, 55 op/s 2024-06-03T01:20:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:32 smithi179 ceph-mon[37041]: pgmap v316: 129 pgs: 129 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 285 KiB/s wr, 55 op/s 2024-06-03T01:20:33.124 INFO:tasks.workunit.client.0.smithi094.stdout: 1 225205 2.03 MB/sec execute 400 sec latency 1174.054 ms 2024-06-03T01:20:34.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 225205 2.02 MB/sec execute 401 sec latency 2174.216 ms 2024-06-03T01:20:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:34 smithi107 ceph-mon[36622]: pgmap v317: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 701 KiB/s wr, 82 op/s 2024-06-03T01:20:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:34 smithi094 ceph-mon[32119]: pgmap v317: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 701 KiB/s wr, 82 op/s 2024-06-03T01:20:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:34 smithi179 ceph-mon[37041]: pgmap v317: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 701 KiB/s wr, 82 op/s 2024-06-03T01:20:35.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 225205 2.01 MB/sec execute 402 sec latency 3174.379 ms 2024-06-03T01:20:35.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:35 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:35.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:35 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:35.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:35 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:36.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 226282 2.02 MB/sec execute 403 sec latency 3371.625 ms 2024-06-03T01:20:36.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:36 smithi107 ceph-mon[36622]: pgmap v318: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 547 KiB/s wr, 52 op/s 2024-06-03T01:20:36.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:36 smithi094 ceph-mon[32119]: pgmap v318: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 547 KiB/s wr, 52 op/s 2024-06-03T01:20:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:36 smithi179 ceph-mon[37041]: pgmap v318: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 547 KiB/s wr, 52 op/s 2024-06-03T01:20:37.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227367 2.04 MB/sec execute 404 sec latency 16.834 ms 2024-06-03T01:20:37.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:37 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:37 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:37.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:37 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:38.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227722 2.03 MB/sec execute 405 sec latency 354.756 ms 2024-06-03T01:20:38.501 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:38 smithi094 ceph-mon[32119]: pgmap v319: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 726 KiB/s wr, 88 op/s 2024-06-03T01:20:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:38 smithi107 ceph-mon[36622]: pgmap v319: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 726 KiB/s wr, 88 op/s 2024-06-03T01:20:38.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:38 smithi179 ceph-mon[37041]: pgmap v319: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 726 KiB/s wr, 88 op/s 2024-06-03T01:20:39.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227722 2.03 MB/sec execute 406 sec latency 1354.898 ms 2024-06-03T01:20:40.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227722 2.02 MB/sec execute 407 sec latency 2355.042 ms 2024-06-03T01:20:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:40 smithi107 ceph-mon[36622]: pgmap v320: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 595 KiB/s wr, 62 op/s 2024-06-03T01:20:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:40 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:40 smithi094 ceph-mon[32119]: pgmap v320: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 595 KiB/s wr, 62 op/s 2024-06-03T01:20:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:40 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:40.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:40 smithi179 ceph-mon[37041]: pgmap v320: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 595 KiB/s wr, 62 op/s 2024-06-03T01:20:40.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:40 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:41.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227778 2.02 MB/sec execute 408 sec latency 3287.159 ms 2024-06-03T01:20:42.125 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227846 2.02 MB/sec execute 409 sec latency 894.470 ms 2024-06-03T01:20:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:42 smithi107 ceph-mon[36622]: pgmap v321: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 595 KiB/s wr, 62 op/s 2024-06-03T01:20:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:42 smithi094 ceph-mon[32119]: pgmap v321: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 595 KiB/s wr, 62 op/s 2024-06-03T01:20:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:42 smithi179 ceph-mon[37041]: pgmap v321: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 595 KiB/s wr, 62 op/s 2024-06-03T01:20:43.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227846 2.01 MB/sec execute 410 sec latency 1894.565 ms 2024-06-03T01:20:44.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227846 2.01 MB/sec execute 411 sec latency 2894.709 ms 2024-06-03T01:20:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:44 smithi107 ceph-mon[36622]: pgmap v322: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 948 KiB/s wr, 92 op/s 2024-06-03T01:20:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:44 smithi094 ceph-mon[32119]: pgmap v322: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 948 KiB/s wr, 92 op/s 2024-06-03T01:20:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:44 smithi179 ceph-mon[37041]: pgmap v322: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 948 KiB/s wr, 92 op/s 2024-06-03T01:20:45.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227846 2.00 MB/sec execute 412 sec latency 3894.837 ms 2024-06-03T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.464+0000 7f4b4ca00640 1 -- 172.21.15.94:0/1698997732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 msgr2=0x7f4b4810b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.464+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/1698997732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 0x7f4b4810b760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009900 tx=0x7f4b3402a480 comp rx=0 tx=0).stop 2024-06-03T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.465+0000 7f4b4ca00640 1 -- 172.21.15.94:0/1698997732 shutdown_connections 2024-06-03T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.465+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/1698997732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b4810bd80 0x7f4b4810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.465+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/1698997732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 0x7f4b4810b760 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.465+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/1698997732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b48102960 0x7f4b48102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.465+0000 7f4b4ca00640 1 -- 172.21.15.94:0/1698997732 >> 172.21.15.94:0/1698997732 conn(0x7f4b480fc1c0 msgr2=0x7f4b480fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.465+0000 7f4b4ca00640 1 -- 172.21.15.94:0/1698997732 shutdown_connections 2024-06-03T01:20:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.465+0000 7f4b4ca00640 1 -- 172.21.15.94:0/1698997732 wait complete. 2024-06-03T01:20:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 Processor -- start 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 -- start start 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b48102960 0x7f4b481980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 0x7f4b48198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b4810bd80 0x7f4b4819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48111b80 con 0x7f4b4810bd80 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48111a00 con 0x7f4b48102960 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.466+0000 7f4b4ca00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48111d00 con 0x7f4b48103280 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 0x7f4b48198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b46800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b48102960 0x7f4b481980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 0x7f4b48198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55714/0 (socket says 172.21.15.94:55714) 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b45e00640 1 -- 172.21.15.94:0/4228297938 learned_addr learned my addr 172.21.15.94:0/4228297938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b47200640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b4810bd80 0x7f4b4819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b46800640 1 -- 172.21.15.94:0/4228297938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 msgr2=0x7f4b48198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b46800640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 0x7f4b48198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b46800640 1 -- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b4810bd80 msgr2=0x7f4b4819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b46800640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b4810bd80 0x7f4b4819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b46800640 1 -- 172.21.15.94:0/4228297938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4819d0b0 con 0x7f4b48102960 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b46800640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b48102960 0x7f4b481980c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e9f0 tx=0x7f4b3800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.467+0000 7f4b3f400640 1 -- 172.21.15.94:0/4228297938 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800ccc0 con 0x7f4b48102960 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.468+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4819d3a0 con 0x7f4b48102960 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.468+0000 7f4b3f400640 1 -- 172.21.15.94:0/4228297938 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b38010910 con 0x7f4b48102960 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.468+0000 7f4b3f400640 1 -- 172.21.15.94:0/4228297938 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38018790 con 0x7f4b48102960 2024-06-03T01:20:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.468+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b481a4c30 con 0x7f4b48102960 2024-06-03T01:20:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 Processor -- start 2024-06-03T01:20:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b3f400640 1 -- 172.21.15.94:0/4228297938 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4b38010ac0 con 0x7f4b48102960 2024-06-03T01:20:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 -- start start 2024-06-03T01:20:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bc200 0x55e95a0bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e95a020420 con 0x55e95a0bd770 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e959fc7c50 con 0x55e95a0bc200 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.471+0000 7f4b50210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e95a01c3e0 con 0x55e95a0cb390 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0bbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60202/0 (socket says 172.21.15.94:60202) 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 -- 172.21.15.94:0/3417640576 learned_addr learned my addr 172.21.15.94:0/3417640576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b45e00640 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b46800640 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bc200 0x55e95a0bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 -- 172.21.15.94:0/3417640576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 msgr2=0x55e95a0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 -- 172.21.15.94:0/3417640576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bc200 msgr2=0x55e95a0bc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bc200 0x55e95a0bc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 -- 172.21.15.94:0/3417640576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e959d34210 con 0x55e95a0bd770 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b45e00640 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b47200640 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0bbcc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009f60 tx=0x7f4b40009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c617ea49836577 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.472+0000 7f4b3f400640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.473+0000 7f4b33400640 1 -- 172.21.15.94:0/3417640576 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4001e070 con 0x55e95a0bd770 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.473+0000 7f4b3f400640 1 -- 172.21.15.94:0/4228297938 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f4b380cfba0 con 0x7f4b48102960 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.473+0000 7f4b33400640 1 -- 172.21.15.94:0/3417640576 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b40003c30 con 0x55e95a0bd770 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.473+0000 7f4b33400640 1 -- 172.21.15.94:0/3417640576 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000f980 con 0x55e95a0bd770 2024-06-03T01:20:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.473+0000 7f4b46800640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 -- 172.21.15.94:0/3417640576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 msgr2=0x55e95a0bbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0bbcc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009f60 tx=0x7f4b40009f90 comp rx=0 tx=0).stop 2024-06-03T01:20:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b46800640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b380ccfe0 tx=0x7f4b380d3420 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 -- 172.21.15.94:0/3417640576 shutdown_connections 2024-06-03T01:20:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bc200 0x55e95a0bc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 --2- 172.21.15.94:0/3417640576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0bbcc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 -- 172.21.15.94:0/3417640576 >> 172.21.15.94:0/3417640576 conn(0x55e95a084f80 msgr2=0x55e95a0b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 -- 172.21.15.94:0/3417640576 shutdown_connections 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 -- 172.21.15.94:0/3417640576 wait complete. 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.474+0000 7f4b50210740 1 -- start start 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 1 Processor -- start 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55e95a0ce160 con 0x55e95a0bc200 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e959fdb0b0 con 0x55e95a0bd770 2024-06-03T01:20:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e95a0ce2e0 con 0x55e95a0cb390 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b50210740 10 client.0 _ll_register_callbacks cb 0x55e959fc6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b47200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b47200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60210/0 (socket says 172.21.15.94:60210) 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.475+0000 7f4b47200640 1 -- 172.21.15.94:0/4288147825 learned_addr learned my addr 172.21.15.94:0/4288147825 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b47200640 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b45e00640 1 -- 172.21.15.94:0/4288147825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 msgr2=0x55e95a0c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b45e00640 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b45e00640 1 -- 172.21.15.94:0/4288147825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 msgr2=0x55e95a0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b45e00640 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b45e00640 1 -- 172.21.15.94:0/4288147825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e959e00620 con 0x55e95a0bd770 2024-06-03T01:20:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b47200640 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b45e00640 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0c8900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009230 tx=0x7f4b34032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91a8b1ed6e08325 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.476+0000 7f4b27e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.477+0000 7f4b27e00640 1 -- 172.21.15.94:0/4288147825 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34035040 con 0x55e95a0bd770 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.477+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b40003c30 con 0x55e95a0bd770 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.477+0000 7f4b27e00640 1 -- 172.21.15.94:0/4288147825 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f4b340318f0 con 0x55e95a0bd770 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.477+0000 7f4b27e00640 1 -- 172.21.15.94:0/4288147825 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f4b3404f050 con 0x55e95a0bd770 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.477+0000 7f4b50210740 10 client.26126 fetch_fsmap learned FSMap version 52 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.477+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e95a0da520 con 0x55e95a0bd770 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.477+0000 7f4b27e00640 1 -- 172.21.15.94:0/4288147825 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b40003c30 con 0x55e95a0bd770 2024-06-03T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b27e00640 1 -- 172.21.15.94:0/4288147825 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7f4b340046c0 con 0x55e95a0bd770 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b50210740 10 client.26126 fetch_fsmap finished waiting for FSMap version 52 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b50210740 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b50210740 20 client.26126 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b50210740 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a0db9e0 0x55e95a0dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b50210740 4 client.26126 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e95a02db50 con 0x55e95a0db9e0 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b47200640 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a0db9e0 0x55e95a0dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b47200640 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a0db9e0 0x55e95a0dde20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f48d0514db08fc server_cookie=3f31d9f946f472be in_seq=0 out_seq=0 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.478+0000 7f4b27e00640 10 client.26126 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b27e00640 1 -- 172.21.15.94:0/4288147825 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e95a02db50 con 0x55e95a0db9e0 2024-06-03T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b27e00640 10 client.26126 handle_command_reply: tid=0 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 client.26126 shutdown 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 msgr2=0x55e95a0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0c8900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009230 tx=0x7f4b34032e60 comp rx=0 tx=0).stop 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 shutdown_connections 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a0db9e0 0x55e95a0dde20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0c8900 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 --2- 172.21.15.94:0/4288147825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 >> 172.21.15.94:0/4288147825 conn(0x55e95a084f80 msgr2=0x55e95a0be5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.479+0000 7f4b27e00640 0 client.26126 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.480+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 shutdown_connections 2024-06-03T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.480+0000 7f4b50210740 1 -- 172.21.15.94:0/4288147825 wait complete. 2024-06-03T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.480+0000 7f4b50210740 20 client.26126 trim_cache size 0 max 16384 2024-06-03T01:20:45.486 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:45 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 Processor -- start 2024-06-03T01:20:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 -- start start 2024-06-03T01:20:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481acb00 con 0x55e95a0bc200 2024-06-03T01:20:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481ae6d0 con 0x55e95a0bd770 2024-06-03T01:20:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.497+0000 7f4b50210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481ad1d0 con 0x55e95a0cb390 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b47200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:39014/0 (socket says 172.21.15.94:39014) 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 -- 172.21.15.94:0/1325850863 learned_addr learned my addr 172.21.15.94:0/1325850863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b47200640 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 -- 172.21.15.94:0/1325850863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 msgr2=0x55e95a0b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 -- 172.21.15.94:0/1325850863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 msgr2=0x55e95a0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 -- 172.21.15.94:0/1325850863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95a0c7370 con 0x55e95a0bd770 2024-06-03T01:20:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b45e00640 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0b80f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b34000c00 tx=0x7f4b34031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999acade1c90e8ec server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.498+0000 7f4b32a00640 1 -- 172.21.15.94:0/1325850863 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34057070 con 0x55e95a0bd770 2024-06-03T01:20:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b32a00640 1 -- 172.21.15.94:0/1325850863 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b34005850 con 0x55e95a0bd770 2024-06-03T01:20:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b32a00640 1 -- 172.21.15.94:0/1325850863 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34000e60 con 0x55e95a0bd770 2024-06-03T01:20:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 -- 172.21.15.94:0/1325850863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 msgr2=0x55e95a0b80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0b80f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b34000c00 tx=0x7f4b34031400 comp rx=0 tx=0).stop 2024-06-03T01:20:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 -- 172.21.15.94:0/1325850863 shutdown_connections 2024-06-03T01:20:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0cb390 0x55e95a0b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0bd770 0x55e95a0b80f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 --2- 172.21.15.94:0/1325850863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bc200 0x55e95a0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 -- 172.21.15.94:0/1325850863 >> 172.21.15.94:0/1325850863 conn(0x55e95a084f80 msgr2=0x55e95a0c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 -- 172.21.15.94:0/1325850863 shutdown_connections 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.499+0000 7f4b50210740 1 -- 172.21.15.94:0/1325850863 wait complete. 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.500+0000 7f4b50210740 1 -- start start 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.500+0000 7f4b50210740 1 Processor -- start 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b50210740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0bc200 0x55e95a0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b50210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b50210740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0cb390 0x55e95a0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b50210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481adf70 con 0x55e95a0bd770 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b50210740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481ad8a0 con 0x55e95a0cb390 2024-06-03T01:20:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b50210740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481aab00 con 0x55e95a0bc200 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b50210740 10 client.0 _ll_register_callbacks cb 0x7f4b400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b47200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0bc200 0x55e95a0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b45e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b47200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0bc200 0x55e95a0d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55760/0 (socket says 172.21.15.94:55760) 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b47200640 1 -- 172.21.15.94:0/3523184957 learned_addr learned my addr 172.21.15.94:0/3523184957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.501+0000 7f4b47200640 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0cb390 0x55e95a0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b45e00640 1 -- 172.21.15.94:0/3523184957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0bc200 msgr2=0x55e95a0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b45e00640 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0bc200 0x55e95a0d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b45e00640 1 -- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0cb390 msgr2=0x55e95a0c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b45e00640 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0cb390 0x55e95a0c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b45e00640 1 -- 172.21.15.94:0/3523184957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95a0c5370 con 0x55e95a0bd770 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b47200640 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0bc200 0x55e95a0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b45e00640 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0d3ca0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55e95a0de2a0 tx=0x7f4b340319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29281977eaa07234 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b27e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:20:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b27e00640 1 -- 172.21.15.94:0/3523184957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34057040 con 0x55e95a0bd770 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b34005850 con 0x55e95a0bd770 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b27e00640 1 -- 172.21.15.94:0/3523184957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f4b34004e40 con 0x55e95a0bd770 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b27e00640 1 -- 172.21.15.94:0/3523184957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f4b34035030 con 0x55e95a0bd770 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.502+0000 7f4b50210740 10 client.16020 fetch_fsmap learned FSMap version 52 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e95a146130 con 0x55e95a0bd770 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b27e00640 1 -- 172.21.15.94:0/3523184957 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b34005850 con 0x55e95a0bd770 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b27e00640 1 -- 172.21.15.94:0/3523184957 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7f4b340468a0 con 0x55e95a0bd770 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b50210740 10 client.16020 fetch_fsmap finished waiting for FSMap version 52 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b50210740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b50210740 20 client.16020 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b50210740 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a1467d0 0x55e95a148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b50210740 4 client.16020 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:20:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e95a027ea0 con 0x55e95a1467d0 2024-06-03T01:20:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.503+0000 7f4b47200640 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a1467d0 0x55e95a148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.504+0000 7f4b47200640 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a1467d0 0x55e95a148bf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83a0283f01216e9 server_cookie=c187a5685add5f10 in_seq=0 out_seq=0 2024-06-03T01:20:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.504+0000 7f4b27e00640 10 client.16020 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.525+0000 7f4b27e00640 1 -- 172.21.15.94:0/3523184957 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e95a027ea0 con 0x55e95a1467d0 2024-06-03T01:20:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.525+0000 7f4b27e00640 10 client.16020 handle_command_reply: tid=0 2024-06-03T01:20:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.525+0000 7f4b50210740 1 client.16020 shutdown 2024-06-03T01:20:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 msgr2=0x55e95a0d3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0d3ca0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55e95a0de2a0 tx=0x7f4b340319a0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 shutdown_connections 2024-06-03T01:20:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55e95a1467d0 0x55e95a148bf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e95a0cb390 0x55e95a0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e95a0bd770 0x55e95a0d3ca0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 --2- 172.21.15.94:0/3523184957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e95a0bc200 0x55e95a0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b27e00640 0 client.16020 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 >> 172.21.15.94:0/3523184957 conn(0x55e95a084f80 msgr2=0x55e95a0c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 shutdown_connections 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 1 -- 172.21.15.94:0/3523184957 wait complete. 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.526+0000 7f4b50210740 20 client.16020 trim_cache size 0 max 16384 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:20:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:20:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:20:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4b2c0a2cf0 msgr2=0x7f4b2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b380ccfe0 tx=0x7f4b380d3420 comp rx=0 tx=0).stop 2024-06-03T01:20:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b48102960 msgr2=0x7f4b481980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b48102960 0x7f4b481980c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e9f0 tx=0x7f4b3800edf0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 shutdown_connections 2024-06-03T01:20:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f4b2c0a2cf0 0x7f4b2c0a51b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4b4810bd80 0x7f4b4819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48103280 0x7f4b48198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 --2- 172.21.15.94:0/4228297938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b48102960 0x7f4b481980c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 >> 172.21.15.94:0/4228297938 conn(0x7f4b480fc1c0 msgr2=0x7f4b480fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 shutdown_connections 2024-06-03T01:20:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:45.527+0000 7f4b4ca00640 1 -- 172.21.15.94:0/4228297938 wait complete. 2024-06-03T01:20:45.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:45 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:45.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:45 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:20:46.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 227939 2.00 MB/sec execute 413 sec latency 4821.910 ms 2024-06-03T01:20:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:46 smithi107 ceph-mon[36622]: pgmap v323: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 531 KiB/s wr, 65 op/s 2024-06-03T01:20:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:46 smithi094 ceph-mon[32119]: pgmap v323: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 531 KiB/s wr, 65 op/s 2024-06-03T01:20:46.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:46 smithi179 ceph-mon[37041]: pgmap v323: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 531 KiB/s wr, 65 op/s 2024-06-03T01:20:47.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 228748 2.00 MB/sec execute 414 sec latency 327.663 ms 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.787 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:47 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:47 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:47.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:47 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:48.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 228748 2.00 MB/sec execute 415 sec latency 1327.759 ms 2024-06-03T01:20:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:48 smithi094 ceph-mon[32119]: pgmap v324: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 655 KiB/s wr, 87 op/s 2024-06-03T01:20:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:48 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-06-03T01:20:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:48 smithi094 ceph-mon[32119]: Unable to set osd_memory_target on smithi179 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-06-03T01:20:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:48 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-06-03T01:20:48.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:48 smithi094 ceph-mon[32119]: Unable to set osd_memory_target on smithi107 to 184723660: error parsing value: Value '184723660' is below minimum 939524096 2024-06-03T01:20:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:48 smithi179 ceph-mon[37041]: pgmap v324: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 655 KiB/s wr, 87 op/s 2024-06-03T01:20:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:48 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-06-03T01:20:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:48 smithi179 ceph-mon[37041]: Unable to set osd_memory_target on smithi179 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-06-03T01:20:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:48 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-06-03T01:20:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:48 smithi179 ceph-mon[37041]: Unable to set osd_memory_target on smithi107 to 184723660: error parsing value: Value '184723660' is below minimum 939524096 2024-06-03T01:20:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:48 smithi107 ceph-mon[36622]: pgmap v324: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 655 KiB/s wr, 87 op/s 2024-06-03T01:20:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:48 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-06-03T01:20:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:48 smithi107 ceph-mon[36622]: Unable to set osd_memory_target on smithi179 to 184767385: error parsing value: Value '184767385' is below minimum 939524096 2024-06-03T01:20:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:48 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-06-03T01:20:49.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:48 smithi107 ceph-mon[36622]: Unable to set osd_memory_target on smithi107 to 184723660: error parsing value: Value '184723660' is below minimum 939524096 2024-06-03T01:20:49.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 228748 1.99 MB/sec execute 416 sec latency 2327.892 ms 2024-06-03T01:20:49.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:20:49.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:49 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:20:49.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:20:49.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:49 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:20:50.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:49 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:20:50.126 INFO:tasks.workunit.client.0.smithi094.stdout: 1 228748 1.99 MB/sec execute 417 sec latency 3328.041 ms 2024-06-03T01:20:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:50 smithi094 ceph-mon[32119]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-06-03T01:20:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:50 smithi094 ceph-mon[32119]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-06-03T01:20:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:50 smithi094 ceph-mon[32119]: pgmap v325: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 476 KiB/s wr, 51 op/s 2024-06-03T01:20:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:50 smithi179 ceph-mon[37041]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-06-03T01:20:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:50 smithi179 ceph-mon[37041]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-06-03T01:20:50.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:50 smithi179 ceph-mon[37041]: pgmap v325: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 476 KiB/s wr, 51 op/s 2024-06-03T01:20:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:50 smithi107 ceph-mon[36622]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-06-03T01:20:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:50 smithi107 ceph-mon[36622]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-06-03T01:20:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:50 smithi107 ceph-mon[36622]: pgmap v325: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 476 KiB/s wr, 51 op/s 2024-06-03T01:20:51.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 229829 2.00 MB/sec execute 418 sec latency 3525.013 ms 2024-06-03T01:20:52.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 230971 2.01 MB/sec execute 419 sec latency 16.536 ms 2024-06-03T01:20:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:52 smithi094 ceph-mon[32119]: pgmap v326: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 476 KiB/s wr, 51 op/s 2024-06-03T01:20:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:52 smithi179 ceph-mon[37041]: pgmap v326: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 476 KiB/s wr, 51 op/s 2024-06-03T01:20:53.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:52 smithi107 ceph-mon[36622]: pgmap v326: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 476 KiB/s wr, 51 op/s 2024-06-03T01:20:53.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 231271 2.01 MB/sec execute 420 sec latency 583.102 ms 2024-06-03T01:20:54.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 231271 2.00 MB/sec execute 421 sec latency 1583.252 ms 2024-06-03T01:20:54.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:54 smithi094 ceph-mon[32119]: pgmap v327: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 619 KiB/s wr, 80 op/s 2024-06-03T01:20:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:54 smithi179 ceph-mon[37041]: pgmap v327: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 619 KiB/s wr, 80 op/s 2024-06-03T01:20:55.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:54 smithi107 ceph-mon[36622]: pgmap v327: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 619 KiB/s wr, 80 op/s 2024-06-03T01:20:55.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 231271 2.00 MB/sec execute 422 sec latency 2583.397 ms 2024-06-03T01:20:56.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 231326 1.99 MB/sec execute 423 sec latency 3511.796 ms 2024-06-03T01:20:56.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:56 smithi094 ceph-mon[32119]: pgmap v328: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 266 KiB/s wr, 50 op/s 2024-06-03T01:20:56.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:56 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:56.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:56 smithi179 ceph-mon[37041]: pgmap v328: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 266 KiB/s wr, 50 op/s 2024-06-03T01:20:56.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:56 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:57.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:56 smithi107 ceph-mon[36622]: pgmap v328: 129 pgs: 129 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 266 KiB/s wr, 50 op/s 2024-06-03T01:20:57.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:56 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:20:57.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 232296 2.00 MB/sec execute 424 sec latency 100.815 ms 2024-06-03T01:20:58.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 233483 2.01 MB/sec execute 425 sec latency 352.233 ms 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 -- 172.21.15.94:0/2317733299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 msgr2=0x7f5b541145b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 --2- 172.21.15.94:0/2317733299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b541145b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009900 tx=0x7f5b3c02a470 comp rx=0 tx=0).stop 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 -- 172.21.15.94:0/2317733299 shutdown_connections 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 --2- 172.21.15.94:0/2317733299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b54114af0 0x7f5b54116ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 --2- 172.21.15.94:0/2317733299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b541145b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 --2- 172.21.15.94:0/2317733299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541089a0 0x7f5b54108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 -- 172.21.15.94:0/2317733299 >> 172.21.15.94:0/2317733299 conn(0x7f5b5406cf00 msgr2=0x7f5b5406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.667+0000 7f5b5a000640 1 -- 172.21.15.94:0/2317733299 shutdown_connections 2024-06-03T01:20:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.668+0000 7f5b5a000640 1 -- 172.21.15.94:0/2317733299 wait complete. 2024-06-03T01:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.668+0000 7f5b5a000640 1 Processor -- start 2024-06-03T01:20:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b5a000640 1 -- start start 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b5a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541089a0 0x7f5b5410a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b5a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b5410a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b5a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b54114af0 0x7f5b5410ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b5a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5411a560 con 0x7f5b541089a0 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b5a000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5411a3e0 con 0x7f5b54114af0 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b5a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5411a6e0 con 0x7f5b541092c0 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b5410a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b5410a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55486/0 (socket says 172.21.15.94:55486) 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b4aa00640 1 -- 172.21.15.94:0/3477678731 learned_addr learned my addr 172.21.15.94:0/3477678731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b53400640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541089a0 0x7f5b5410a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.669+0000 7f5b53e00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b54114af0 0x7f5b5410ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b53e00640 1 -- 172.21.15.94:0/3477678731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 msgr2=0x7f5b5410a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b53e00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b5410a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b53e00640 1 -- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541089a0 msgr2=0x7f5b5410a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b53e00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541089a0 0x7f5b5410a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b53e00640 1 -- 172.21.15.94:0/3477678731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b541aeee0 con 0x7f5b54114af0 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b53400640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541089a0 0x7f5b5410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b5410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b53e00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b54114af0 0x7f5b5410ae90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005fd0 tx=0x7f5b4c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b50c00640 1 -- 172.21.15.94:0/3477678731 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00cdf0 con 0x7f5b54114af0 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b50c00640 1 -- 172.21.15.94:0/3477678731 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4c005020 con 0x7f5b54114af0 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b50c00640 1 -- 172.21.15.94:0/3477678731 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0106d0 con 0x7f5b54114af0 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.670+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b541af1d0 con 0x7f5b54114af0 2024-06-03T01:20:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.671+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541af6d0 con 0x7f5b54114af0 2024-06-03T01:20:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.674+0000 7f5b50c00640 1 -- 172.21.15.94:0/3477678731 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5b4c0054d0 con 0x7f5b54114af0 2024-06-03T01:20:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.674+0000 7f5b5cdd1740 1 Processor -- start 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.674+0000 7f5b5cdd1740 1 -- start start 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb045b0 0x559dccb02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb03040 0x559dccb034c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb0ddc0 0x559dccb101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559dcca67420 con 0x559dccb045b0 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559dcca0ec50 con 0x559dccb0ddc0 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559dcca633e0 con 0x559dccb03040 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb03040 0x559dccb034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb045b0 0x559dccb02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb03040 0x559dccb034c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55488/0 (socket says 172.21.15.94:55488) 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 -- 172.21.15.94:0/900300125 learned_addr learned my addr 172.21.15.94:0/900300125 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb0ddc0 0x559dccb101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 -- 172.21.15.94:0/900300125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb0ddc0 msgr2=0x559dccb101b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb0ddc0 0x559dccb101b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 -- 172.21.15.94:0/900300125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb045b0 msgr2=0x559dccb02b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb045b0 0x559dccb02b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b53400640 1 -- 172.21.15.94:0/900300125 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcc77b210 con 0x559dccb03040 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.675+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb045b0 0x559dccb02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.676+0000 7f5b50c00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5b380a2e20 0x7f5b380a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.676+0000 7f5b50c00640 1 -- 172.21.15.94:0/3477678731 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f5b4c0cfe30 con 0x7f5b54114af0 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.676+0000 7f5b53400640 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb03040 0x559dccb034c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b44011620 tx=0x7f5b44011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679419d23a079f31 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.676+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5b380a2e20 0x7f5b380a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.676+0000 7f5b43e00640 1 -- 172.21.15.94:0/900300125 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4401e070 con 0x559dccb03040 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.676+0000 7f5b43e00640 1 -- 172.21.15.94:0/900300125 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4400fb40 con 0x559dccb03040 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.676+0000 7f5b43e00640 1 -- 172.21.15.94:0/900300125 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44019cf0 con 0x559dccb03040 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/900300125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb03040 msgr2=0x559dccb034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5b380a2e20 0x7f5b380a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c031550 tx=0x7f5b3c005a20 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb03040 0x559dccb034c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b44011620 tx=0x7f5b44011ae0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/900300125 shutdown_connections 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb0ddc0 0x559dccb101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb03040 0x559dccb034c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/900300125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb045b0 0x559dccb02b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/900300125 >> 172.21.15.94:0/900300125 conn(0x559dccacbf80 msgr2=0x559dccaff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/900300125 shutdown_connections 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.677+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/900300125 wait complete. 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.678+0000 7f5b5cdd1740 1 -- start start 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.678+0000 7f5b5cdd1740 1 Processor -- start 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.678+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb0adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.678+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb0b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.678+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb0b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.678+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x559dccb10b90 con 0x559dccb0ddc0 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559dcca220b0 con 0x559dccb03040 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559dccb10d10 con 0x559dccb045b0 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b5cdd1740 10 client.0 _ll_register_callbacks cb 0x559dcca0df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb0adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb0adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:39966/0 (socket says 172.21.15.94:39966) 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 -- 172.21.15.94:0/477612808 learned_addr learned my addr 172.21.15.94:0/477612808 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb0b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53e00640 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb0b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 -- 172.21.15.94:0/477612808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 msgr2=0x559dccb0b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb0b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 -- 172.21.15.94:0/477612808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 msgr2=0x559dccb0b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb0b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 -- 172.21.15.94:0/477612808 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcc847620 con 0x559dccb03040 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b53400640 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb0adc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b44019830 tx=0x7f5b44019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e4171bcfc1f5aca server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.679+0000 7f5b40c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b40c00640 1 -- 172.21.15.94:0/477612808 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44027060 con 0x559dccb03040 2024-06-03T01:20:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b40c00640 1 -- 172.21.15.94:0/477612808 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f5b4400f660 con 0x559dccb03040 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4400fb40 con 0x559dccb03040 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b40c00640 1 -- 172.21.15.94:0/477612808 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5b4401e040 con 0x559dccb03040 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b5cdd1740 10 client.26144 fetch_fsmap learned FSMap version 53 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x559dccb1ced0 con 0x559dccb03040 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b40c00640 1 -- 172.21.15.94:0/477612808 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4400fb40 con 0x559dccb03040 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b40c00640 1 -- 172.21.15.94:0/477612808 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 53) ==== 8378+0+0 (secure 0 0 0) 0x7f5b44021030 con 0x559dccb03040 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b5cdd1740 10 client.26144 fetch_fsmap finished waiting for FSMap version 53 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.680+0000 7f5b5cdd1740 10 client.26144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.681+0000 7f5b5cdd1740 20 client.26144 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.681+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb1e2d0 0x559dccb206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.681+0000 7f5b5cdd1740 4 client.26144 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.681+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dcca74b50 con 0x559dccb1e2d0 2024-06-03T01:20:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.681+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb1e2d0 0x559dccb206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.681+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb1e2d0 0x559dccb206c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb5da815b64e0d1 server_cookie=a246e6b7b0914b30 in_seq=0 out_seq=0 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.681+0000 7f5b40c00640 10 client.26144 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b40c00640 1 -- 172.21.15.94:0/477612808 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559dcca74b50 con 0x559dccb1e2d0 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b40c00640 10 client.26144 handle_command_reply: tid=0 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b5cdd1740 1 client.26144 shutdown 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 msgr2=0x559dccb0adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb0adc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b44019830 tx=0x7f5b44019640 comp rx=0 tx=0).stop 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 shutdown_connections 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb0b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb0b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.682+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb1e2d0 0x559dccb206c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.683+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/477612808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb0adc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.683+0000 7f5b40c00640 0 client.26144 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.683+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 >> 172.21.15.94:0/477612808 conn(0x559dccacbf80 msgr2=0x559dccb05460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.683+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 shutdown_connections 2024-06-03T01:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.683+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/477612808 wait complete. 2024-06-03T01:20:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.683+0000 7f5b5cdd1740 20 client.26144 trim_cache size 0 max 16384 2024-06-03T01:20:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 Processor -- start 2024-06-03T01:20:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 -- start start 2024-06-03T01:20:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb03040 0x559dccafe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb045b0 0x559dccb25ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb0ddc0 0x559dccb26400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541b98d0 con 0x559dccb03040 2024-06-03T01:20:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541bb4d0 con 0x559dccb045b0 2024-06-03T01:20:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.700+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541b9fa0 con 0x559dccb0ddc0 2024-06-03T01:20:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb03040 0x559dccafe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb045b0 0x559dccb25ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb03040 0x559dccafe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41452/0 (socket says 172.21.15.94:41452) 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 -- 172.21.15.94:0/740569861 learned_addr learned my addr 172.21.15.94:0/740569861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53e00640 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb0ddc0 0x559dccb26400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 -- 172.21.15.94:0/740569861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb0ddc0 msgr2=0x559dccb26400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb0ddc0 0x559dccb26400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 -- 172.21.15.94:0/740569861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb045b0 msgr2=0x559dccb25ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb045b0 0x559dccb25ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b53400640 1 -- 172.21.15.94:0/740569861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dccb20a30 con 0x559dccb03040 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.701+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb045b0 0x559dccb25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.702+0000 7f5b53400640 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb03040 0x559dccafe690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5b44011b20 tx=0x7f5b440049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e124eca8f9d082e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.702+0000 7f5b43400640 1 -- 172.21.15.94:0/740569861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44027040 con 0x559dccb03040 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.702+0000 7f5b43400640 1 -- 172.21.15.94:0/740569861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4400fb40 con 0x559dccb03040 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.702+0000 7f5b43400640 1 -- 172.21.15.94:0/740569861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440042a0 con 0x559dccb03040 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/740569861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb03040 msgr2=0x559dccafe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb03040 0x559dccafe690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5b44011b20 tx=0x7f5b440049d0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/740569861 shutdown_connections 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb0ddc0 0x559dccb26400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb045b0 0x559dccb25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/740569861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb03040 0x559dccafe690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/740569861 >> 172.21.15.94:0/740569861 conn(0x559dccacbf80 msgr2=0x559dccafdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/740569861 shutdown_connections 2024-06-03T01:20:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/740569861 wait complete. 2024-06-03T01:20:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 -- start start 2024-06-03T01:20:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.703+0000 7f5b5cdd1740 1 Processor -- start 2024-06-03T01:20:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb16370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b5cdd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb11820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541bad70 con 0x559dccb0ddc0 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541ba6a0 con 0x559dccb03040 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b5cdd1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541b78d0 con 0x559dccb045b0 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b5cdd1740 10 client.0 _ll_register_callbacks cb 0x7f5b4401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb16370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb16370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:39978/0 (socket says 172.21.15.94:39978) 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 -- 172.21.15.94:0/1826137346 learned_addr learned my addr 172.21.15.94:0/1826137346 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53e00640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb11820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 -- 172.21.15.94:0/1826137346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 msgr2=0x559dccb168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 -- 172.21.15.94:0/1826137346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 msgr2=0x559dccb11820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:20:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb11820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53400640 1 -- 172.21.15.94:0/1826137346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dccb12050 con 0x559dccb03040 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b53e00640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb11820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.704+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b53400640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb16370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005660 tx=0x7f5b44004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c17776d4102f75 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b40c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b40c00640 1 -- 172.21.15.94:0/1826137346 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44027040 con 0x559dccb03040 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b40c00640 1 -- 172.21.15.94:0/1826137346 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9475+0+0 (secure 0 0 0) 0x7f5b44019830 con 0x559dccb03040 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4400fb40 con 0x559dccb03040 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b40c00640 1 -- 172.21.15.94:0/1826137346 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5b44045050 con 0x559dccb03040 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b5cdd1740 10 client.26150 fetch_fsmap learned FSMap version 53 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.705+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x559dccb88b00 con 0x559dccb03040 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b40c00640 1 -- 172.21.15.94:0/1826137346 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4400fb40 con 0x559dccb03040 2024-06-03T01:20:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b40c00640 1 -- 172.21.15.94:0/1826137346 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 53) ==== 8378+0+0 (secure 0 0 0) 0x7f5b44021070 con 0x559dccb03040 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b5cdd1740 10 client.26150 fetch_fsmap finished waiting for FSMap version 53 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b5cdd1740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b5cdd1740 20 client.26150 populate_metadata read hostname 'smithi094' 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb89260 0x559dccb8b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b5cdd1740 4 client.26150 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dcca6eea0 con 0x559dccb89260 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.706+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb89260 0x559dccb8b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.707+0000 7f5b4aa00640 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb89260 0x559dccb8b680 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994fe43d32e5db15 server_cookie=2bc6187c05cb380b in_seq=0 out_seq=0 2024-06-03T01:20:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.707+0000 7f5b40c00640 10 client.26150 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b40c00640 1 -- 172.21.15.94:0/1826137346 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559dcca6eea0 con 0x559dccb89260 2024-06-03T01:20:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b40c00640 10 client.26150 handle_command_reply: tid=0 2024-06-03T01:20:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 client.26150 shutdown 2024-06-03T01:20:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 msgr2=0x559dccb16370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb16370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005660 tx=0x7f5b44004270 comp rx=0 tx=0).stop 2024-06-03T01:20:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 shutdown_connections 2024-06-03T01:20:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dccb0ddc0 0x559dccb11820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dccb045b0 0x559dccb168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x559dccb89260 0x559dccb8b680 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.726+0000 7f5b5cdd1740 1 --2- 172.21.15.94:0/1826137346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dccb03040 0x559dccb16370 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.727+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 >> 172.21.15.94:0/1826137346 conn(0x559dccacbf80 msgr2=0x559dccafdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.727+0000 7f5b40c00640 0 client.26150 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.727+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 shutdown_connections 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.727+0000 7f5b5cdd1740 1 -- 172.21.15.94:0/1826137346 wait complete. 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.727+0000 7f5b5cdd1740 20 client.26150 trim_cache size 0 max 16384 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:20:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:20:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.728+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5b380a2e20 msgr2=0x7f5b380a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.728+0000 7f5b5a000640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5b380a2e20 0x7f5b380a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c031550 tx=0x7f5b3c005a20 comp rx=0 tx=0).stop 2024-06-03T01:20:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.728+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b54114af0 msgr2=0x7f5b5410ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:20:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.728+0000 7f5b5a000640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b54114af0 0x7f5b5410ae90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005fd0 tx=0x7f5b4c00ef10 comp rx=0 tx=0).stop 2024-06-03T01:20:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 shutdown_connections 2024-06-03T01:20:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5b380a2e20 0x7f5b380a52e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b54114af0 0x7f5b5410ae90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b541092c0 0x7f5b5410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 --2- 172.21.15.94:0/3477678731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b541089a0 0x7f5b5410a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:20:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 >> 172.21.15.94:0/3477678731 conn(0x7f5b5406cf00 msgr2=0x7f5b540705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:20:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 shutdown_connections 2024-06-03T01:20:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:20:58.729+0000 7f5b5a000640 1 -- 172.21.15.94:0/3477678731 wait complete. 2024-06-03T01:20:58.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:20:58 smithi179 ceph-mon[37041]: pgmap v329: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 685 KiB/s wr, 88 op/s 2024-06-03T01:20:59.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:20:58 smithi107 ceph-mon[36622]: pgmap v329: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 685 KiB/s wr, 88 op/s 2024-06-03T01:20:59.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:20:58 smithi094 ceph-mon[32119]: pgmap v329: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 685 KiB/s wr, 88 op/s 2024-06-03T01:20:59.127 INFO:tasks.workunit.client.0.smithi094.stdout: 1 234553 2.02 MB/sec execute 426 sec latency 15.735 ms 2024-06-03T01:21:00.128 INFO:tasks.workunit.client.0.smithi094.stdout: 1 234814 2.02 MB/sec execute 427 sec latency 626.444 ms 2024-06-03T01:21:01.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:00 smithi107 ceph-mon[36622]: pgmap v330: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 561 KiB/s wr, 65 op/s 2024-06-03T01:21:01.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:00 smithi094 ceph-mon[32119]: pgmap v330: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 561 KiB/s wr, 65 op/s 2024-06-03T01:21:01.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:00 smithi179 ceph-mon[37041]: pgmap v330: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 561 KiB/s wr, 65 op/s 2024-06-03T01:21:01.128 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235156 2.02 MB/sec execute 428 sec latency 999.830 ms 2024-06-03T01:21:01.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:01 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:02.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:01 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:01 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:02.128 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235182 2.01 MB/sec execute 429 sec latency 113.118 ms 2024-06-03T01:21:03.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:02 smithi107 ceph-mon[36622]: pgmap v331: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 561 KiB/s wr, 65 op/s 2024-06-03T01:21:03.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:02 smithi179 ceph-mon[37041]: pgmap v331: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 561 KiB/s wr, 65 op/s 2024-06-03T01:21:03.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:02 smithi094 ceph-mon[32119]: pgmap v331: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 561 KiB/s wr, 65 op/s 2024-06-03T01:21:03.352 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235839 2.02 MB/sec execute 430 sec latency 323.711 ms 2024-06-03T01:21:04.128 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235839 2.01 MB/sec execute 431 sec latency 1323.856 ms 2024-06-03T01:21:05.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:04 smithi107 ceph-mon[36622]: pgmap v332: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 970 KiB/s wr, 143 op/s 2024-06-03T01:21:05.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:04 smithi107 ceph-mon[36622]: osdmap e110: 12 total, 12 up, 12 in 2024-06-03T01:21:05.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:04 smithi094 ceph-mon[32119]: pgmap v332: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 970 KiB/s wr, 143 op/s 2024-06-03T01:21:05.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:04 smithi094 ceph-mon[32119]: osdmap e110: 12 total, 12 up, 12 in 2024-06-03T01:21:05.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:04 smithi179 ceph-mon[37041]: pgmap v332: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 970 KiB/s wr, 143 op/s 2024-06-03T01:21:05.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:04 smithi179 ceph-mon[37041]: osdmap e110: 12 total, 12 up, 12 in 2024-06-03T01:21:05.128 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235839 2.01 MB/sec execute 432 sec latency 2323.993 ms 2024-06-03T01:21:06.128 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235839 2.00 MB/sec execute 433 sec latency 3324.135 ms 2024-06-03T01:21:07.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:06 smithi107 ceph-mon[36622]: pgmap v334: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 992 KiB/s wr, 137 op/s 2024-06-03T01:21:07.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:06 smithi094 ceph-mon[32119]: pgmap v334: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 992 KiB/s wr, 137 op/s 2024-06-03T01:21:07.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:06 smithi179 ceph-mon[37041]: pgmap v334: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 992 KiB/s wr, 137 op/s 2024-06-03T01:21:07.128 INFO:tasks.workunit.client.0.smithi094.stdout: 1 235839 2.00 MB/sec execute 434 sec latency 4324.267 ms 2024-06-03T01:21:08.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 236755 2.00 MB/sec execute 435 sec latency 4577.053 ms 2024-06-03T01:21:08.902 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:08 smithi094 ceph-mon[32119]: pgmap v335: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 696 KiB/s wr, 121 op/s 2024-06-03T01:21:09.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:08 smithi107 ceph-mon[36622]: pgmap v335: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 696 KiB/s wr, 121 op/s 2024-06-03T01:21:09.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:08 smithi179 ceph-mon[37041]: pgmap v335: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 696 KiB/s wr, 121 op/s 2024-06-03T01:21:09.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 238012 2.02 MB/sec execute 436 sec latency 18.981 ms 2024-06-03T01:21:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:09 smithi107 ceph-mon[36622]: osdmap e111: 12 total, 12 up, 12 in 2024-06-03T01:21:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:09 smithi107 ceph-mon[36622]: pgmap v337: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 870 KiB/s wr, 151 op/s 2024-06-03T01:21:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:09 smithi094 ceph-mon[32119]: osdmap e111: 12 total, 12 up, 12 in 2024-06-03T01:21:10.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:09 smithi094 ceph-mon[32119]: pgmap v337: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 870 KiB/s wr, 151 op/s 2024-06-03T01:21:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:09 smithi179 ceph-mon[37041]: osdmap e111: 12 total, 12 up, 12 in 2024-06-03T01:21:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:09 smithi179 ceph-mon[37041]: pgmap v337: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 870 KiB/s wr, 151 op/s 2024-06-03T01:21:10.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 238363 2.02 MB/sec execute 437 sec latency 453.060 ms 2024-06-03T01:21:11.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 238419 2.01 MB/sec execute 438 sec latency 1377.571 ms 2024-06-03T01:21:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 -- 172.21.15.94:0/3763501529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 msgr2=0x7f54b4114ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 --2- 172.21.15.94:0/3763501529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 0x7f54b4114ba0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009860 tx=0x7f54ac02a3d0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 -- 172.21.15.94:0/3763501529 shutdown_connections 2024-06-03T01:21:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 --2- 172.21.15.94:0/3763501529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 0x7f54b4114ba0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 --2- 172.21.15.94:0/3763501529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b4074660 0x7f54b4112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 --2- 172.21.15.94:0/3763501529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 0x7f54b4074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 -- 172.21.15.94:0/3763501529 >> 172.21.15.94:0/3763501529 conn(0x7f54b40fd2f0 msgr2=0x7f54b40ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 -- 172.21.15.94:0/3763501529 shutdown_connections 2024-06-03T01:21:11.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.876+0000 7f54bae00640 1 -- 172.21.15.94:0/3763501529 wait complete. 2024-06-03T01:21:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.877+0000 7f54bae00640 1 Processor -- start 2024-06-03T01:21:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.877+0000 7f54bae00640 1 -- start start 2024-06-03T01:21:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54bae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 0x7f54b419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54bae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b4074660 0x7f54b419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54bae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 0x7f54b41a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54bae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4118150 con 0x7f54b4073d40 2024-06-03T01:21:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54bae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4117fd0 con 0x7f54b4074660 2024-06-03T01:21:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54bae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41182d0 con 0x7f54b41127b0 2024-06-03T01:21:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 0x7f54b419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54ba400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 0x7f54b41a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 0x7f54b419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59126/0 (socket says 172.21.15.94:59126) 2024-06-03T01:21:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 -- 172.21.15.94:0/241179452 learned_addr learned my addr 172.21.15.94:0/241179452 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9000640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b4074660 0x7f54b419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 -- 172.21.15.94:0/241179452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 msgr2=0x7f54b41a2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 0x7f54b41a2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 -- 172.21.15.94:0/241179452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b4074660 msgr2=0x7f54b419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b4074660 0x7f54b419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.878+0000 7f54b9a00640 1 -- 172.21.15.94:0/241179452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b41a3670 con 0x7f54b4073d40 2024-06-03T01:21:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.879+0000 7f54b9a00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 0x7f54b419e6d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009d50 tx=0x7f54a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:11.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.879+0000 7f54b2000640 1 -- 172.21.15.94:0/241179452 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4003b70 con 0x7f54b4073d40 2024-06-03T01:21:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.879+0000 7f54b2000640 1 -- 172.21.15.94:0/241179452 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a4003d10 con 0x7f54b4073d40 2024-06-03T01:21:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.879+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b41a3960 con 0x7f54b4073d40 2024-06-03T01:21:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.879+0000 7f54b2000640 1 -- 172.21.15.94:0/241179452 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4010a20 con 0x7f54b4073d40 2024-06-03T01:21:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.879+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b41ab1f0 con 0x7f54b4073d40 2024-06-03T01:21:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 Processor -- start 2024-06-03T01:21:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 -- start start 2024-06-03T01:21:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54b2000640 1 -- 172.21.15.94:0/241179452 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f54a4003ec0 con 0x7f54b4073d40 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd7200 0x564ccacd7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccace1f80 0x564ccace4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564ccac3b420 con 0x564ccacd7200 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ccabe2c50 con 0x564ccacd8770 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ccac373e0 con 0x564ccace1f80 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54ba400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54ba400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40500/0 (socket says 172.21.15.94:40500) 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.883+0000 7f54ba400640 1 -- 172.21.15.94:0/935187219 learned_addr learned my addr 172.21.15.94:0/935187219 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54b9000640 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccace1f80 0x564ccace4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54b9a00640 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd7200 0x564ccacd7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54ba400640 1 -- 172.21.15.94:0/935187219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccace1f80 msgr2=0x564ccace4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54ba400640 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccace1f80 0x564ccace4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54ba400640 1 -- 172.21.15.94:0/935187219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd7200 msgr2=0x564ccacd7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54ba400640 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd7200 0x564ccacd7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54ba400640 1 -- 172.21.15.94:0/935187219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cca94f210 con 0x564ccacd8770 2024-06-03T01:21:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54ba400640 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd6cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54ac034000 tx=0x7f54ac02f380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7471d5d069533f75 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54aaa00640 1 -- 172.21.15.94:0/935187219 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac030df0 con 0x564ccacd8770 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.884+0000 7f54b2000640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f54980a2d70 0x7f54980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.885+0000 7f54aaa00640 1 -- 172.21.15.94:0/935187219 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ac0303a0 con 0x564ccacd8770 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.885+0000 7f54aaa00640 1 -- 172.21.15.94:0/935187219 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac005420 con 0x564ccacd8770 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.885+0000 7f54b2000640 1 -- 172.21.15.94:0/241179452 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7f54a40cfea0 con 0x7f54b4073d40 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.885+0000 7f54b9a00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f54980a2d70 0x7f54980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.885+0000 7f54bf3dc740 1 -- 172.21.15.94:0/935187219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 msgr2=0x564ccacd6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.885+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd6cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54ac034000 tx=0x7f54ac02f380 comp rx=0 tx=0).stop 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.885+0000 7f54b9a00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f54980a2d70 0x7f54980a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x564ccacd9ed0 tx=0x7f54a40104c0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 -- 172.21.15.94:0/935187219 shutdown_connections 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccace1f80 0x564ccace4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd7200 0x564ccacd7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/935187219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd6cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 -- 172.21.15.94:0/935187219 >> 172.21.15.94:0/935187219 conn(0x564ccac9ff80 msgr2=0x564ccacd37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 -- 172.21.15.94:0/935187219 shutdown_connections 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 -- 172.21.15.94:0/935187219 wait complete. 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 -- start start 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.886+0000 7f54bf3dc740 1 Processor -- start 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd8770 0x564ccacdf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccace1f80 0x564ccacdfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x564ccace4cf0 con 0x564ccacd8770 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ccabf60b0 con 0x564ccace1f80 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ccace4e70 con 0x564ccacd7200 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54bf3dc740 10 client.0 _ll_register_callbacks cb 0x564ccabe1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54b9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd8770 0x564ccacdf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54ba400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54b9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd8770 0x564ccacdf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59154/0 (socket says 172.21.15.94:59154) 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54b9000640 1 -- 172.21.15.94:0/3464993981 learned_addr learned my addr 172.21.15.94:0/3464993981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.887+0000 7f54ba400640 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccace1f80 0x564ccacdfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54b9000640 1 -- 172.21.15.94:0/3464993981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 msgr2=0x564ccacdef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54b9000640 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54b9000640 1 -- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccace1f80 msgr2=0x564ccacdfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54b9000640 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccace1f80 0x564ccacdfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54b9000640 1 -- 172.21.15.94:0/3464993981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ccaa1b620 con 0x564ccacd8770 2024-06-03T01:21:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54ba400640 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccace1f80 0x564ccacdfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:21:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54b9000640 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd8770 0x564ccacdf4c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f54a00126d0 tx=0x7f54a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfb004f87ba379d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f549f400640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f549f400640 1 -- 172.21.15.94:0/3464993981 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0004020 con 0x564ccacd8770 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f549f400640 1 -- 172.21.15.94:0/3464993981 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7f54a00041c0 con 0x564ccacd8770 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ac0303a0 con 0x564ccacd8770 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.888+0000 7f549f400640 1 -- 172.21.15.94:0/3464993981 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f54a0024050 con 0x564ccacd8770 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 10 client.16050 fetch_fsmap learned FSMap version 54 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564ccacf10c0 con 0x564ccacd8770 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f549f400640 1 -- 172.21.15.94:0/3464993981 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ac0303a0 con 0x564ccacd8770 2024-06-03T01:21:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f549f400640 1 -- 172.21.15.94:0/3464993981 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7f54a0003b40 con 0x564ccacd8770 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 10 client.16050 fetch_fsmap finished waiting for FSMap version 54 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 20 client.16050 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccacf2640 0x564ccacf4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 4 client.16050 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.889+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ccac48b50 con 0x564ccacf2640 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.890+0000 7f54ba400640 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccacf2640 0x564ccacf4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.890+0000 7f54ba400640 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccacf2640 0x564ccacf4a50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426ac38fd528ee server_cookie=b5cd20a105009936 in_seq=0 out_seq=0 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.890+0000 7f549f400640 10 client.16050 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f549f400640 1 -- 172.21.15.94:0/3464993981 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ccac48b50 con 0x564ccacf2640 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f549f400640 10 client.16050 handle_command_reply: tid=0 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 client.16050 shutdown 2024-06-03T01:21:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd8770 msgr2=0x564ccacdf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd8770 0x564ccacdf4c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f54a00126d0 tx=0x7f54a0009f90 comp rx=0 tx=0).stop 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 shutdown_connections 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccacf2640 0x564ccacf4a50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccace1f80 0x564ccacdfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccacd8770 0x564ccacdf4c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/3464993981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f549f400640 0 client.16050 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 >> 172.21.15.94:0/3464993981 conn(0x564ccac9ff80 msgr2=0x564ccacd9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 shutdown_connections 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.891+0000 7f54bf3dc740 1 -- 172.21.15.94:0/3464993981 wait complete. 2024-06-03T01:21:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.892+0000 7f54bf3dc740 20 client.16050 trim_cache size 0 max 16384 2024-06-03T01:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 Processor -- start 2024-06-03T01:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 -- start start 2024-06-03T01:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccacd3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b52c0 con 0x564ccace1f80 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b6f20 con 0x564ccacd8770 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b5990 con 0x564ccacd7200 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54b9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54b9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40526/0 (socket says 172.21.15.94:40526) 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54b9000640 1 -- 172.21.15.94:0/2422616223 learned_addr learned my addr 172.21.15.94:0/2422616223 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.910+0000 7f54ba400640 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54ba400640 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccacd3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54b9000640 1 -- 172.21.15.94:0/2422616223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 msgr2=0x564ccacdf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54b9000640 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54b9000640 1 -- 172.21.15.94:0/2422616223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 msgr2=0x564ccacd3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54b9000640 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccacd3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54b9000640 1 -- 172.21.15.94:0/2422616223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ccaccf1c0 con 0x564ccacd8770 2024-06-03T01:21:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54ba400640 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccacd3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:21:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54b9000640 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd30f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009530 tx=0x7f54a0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a85e6d58fd653d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54aa000640 1 -- 172.21.15.94:0/2422616223 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0000e60 con 0x564ccacd8770 2024-06-03T01:21:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54aa000640 1 -- 172.21.15.94:0/2422616223 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a0021040 con 0x564ccacd8770 2024-06-03T01:21:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.911+0000 7f54aa000640 1 -- 172.21.15.94:0/2422616223 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0004220 con 0x564ccacd8770 2024-06-03T01:21:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2422616223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 msgr2=0x564ccacd30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd30f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009530 tx=0x7f54a0000bc0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2422616223 shutdown_connections 2024-06-03T01:21:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccacd3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacd30f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2422616223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacdf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2422616223 >> 172.21.15.94:0/2422616223 conn(0x564ccac9ff80 msgr2=0x564ccacd95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2422616223 shutdown_connections 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.912+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2422616223 wait complete. 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 -- start start 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 Processor -- start 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccace56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b67c0 con 0x564ccace1f80 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b60f0 con 0x564ccacd8770 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54b41b32c0 con 0x564ccacd7200 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54bf3dc740 10 client.0 _ll_register_callbacks cb 0x7f54a0009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54b9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54b9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacea6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:40536/0 (socket says 172.21.15.94:40536) 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54b9000640 1 -- 172.21.15.94:0/2531402715 learned_addr learned my addr 172.21.15.94:0/2531402715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.913+0000 7f54ba400640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54ba400640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccace56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54b9000640 1 -- 172.21.15.94:0/2531402715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 msgr2=0x564ccacea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54b9000640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54b9000640 1 -- 172.21.15.94:0/2531402715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 msgr2=0x564ccace56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54b9000640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccace56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54b9000640 1 -- 172.21.15.94:0/2531402715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ccace5ec0 con 0x564ccacd8770 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54ba400640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54b9000640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacea6b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54a00058c0 tx=0x7f54a0005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29bd90c81217b39 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f549f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:21:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f549f400640 1 -- 172.21.15.94:0/2531402715 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0035020 con 0x564ccacd8770 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f549f400640 1 -- 172.21.15.94:0/2531402715 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7f54a001c900 con 0x564ccacd8770 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.914+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a0021040 con 0x564ccacd8770 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f549f400640 1 -- 172.21.15.94:0/2531402715 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f54a0004470 con 0x564ccacd8770 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 10 client.26168 fetch_fsmap learned FSMap version 54 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564ccad5cc30 con 0x564ccacd8770 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f549f400640 1 -- 172.21.15.94:0/2531402715 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54a0021040 con 0x564ccacd8770 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f549f400640 1 -- 172.21.15.94:0/2531402715 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7f54a0005200 con 0x564ccacd8770 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 10 client.26168 fetch_fsmap finished waiting for FSMap version 54 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 20 client.26168 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccad5d3d0 0x564ccad5f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 4 client.26168 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.915+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ccac42ea0 con 0x564ccad5d3d0 2024-06-03T01:21:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.916+0000 7f54ba400640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccad5d3d0 0x564ccad5f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.916+0000 7f54ba400640 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccad5d3d0 0x564ccad5f7f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b374dc042c606a6c server_cookie=6c21411b45ef59f6 in_seq=0 out_seq=0 2024-06-03T01:21:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.916+0000 7f549f400640 10 client.26168 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f549f400640 1 -- 172.21.15.94:0/2531402715 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564ccac42ea0 con 0x564ccad5d3d0 2024-06-03T01:21:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f549f400640 10 client.26168 handle_command_reply: tid=0 2024-06-03T01:21:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 client.26168 shutdown 2024-06-03T01:21:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 msgr2=0x564ccacea6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacea6b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54a00058c0 tx=0x7f54a0005c50 comp rx=0 tx=0).stop 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 shutdown_connections 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564ccace1f80 0x564ccace56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x564ccad5d3d0 0x564ccad5f7f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ccacd8770 0x564ccacea6b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 --2- 172.21.15.94:0/2531402715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ccacd7200 0x564ccacea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f549f400640 0 client.26168 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 >> 172.21.15.94:0/2531402715 conn(0x564ccac9ff80 msgr2=0x564ccacecfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 shutdown_connections 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 1 -- 172.21.15.94:0/2531402715 wait complete. 2024-06-03T01:21:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.937+0000 7f54bf3dc740 20 client.26168 trim_cache size 0 max 16384 2024-06-03T01:21:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:21:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:21:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:21:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f54980a2d70 msgr2=0x7f54980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f54980a2d70 0x7f54980a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x564ccacd9ed0 tx=0x7f54a40104c0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 msgr2=0x7f54b419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 0x7f54b419e6d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009d50 tx=0x7f54a400c6a0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 shutdown_connections 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f54980a2d70 0x7f54980a5230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54b41127b0 0x7f54b41a2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54b4074660 0x7f54b419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 --2- 172.21.15.94:0/241179452 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54b4073d40 0x7f54b419e6d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.938+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 >> 172.21.15.94:0/241179452 conn(0x7f54b40fd2f0 msgr2=0x7f54b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.939+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 shutdown_connections 2024-06-03T01:21:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:11.939+0000 7f54bae00640 1 -- 172.21.15.94:0/241179452 wait complete. 2024-06-03T01:21:12.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 239388 2.02 MB/sec execute 439 sec latency 110.245 ms 2024-06-03T01:21:12.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:12 smithi094 ceph-mon[32119]: pgmap v338: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 258 KiB/s wr, 36 op/s 2024-06-03T01:21:12.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:12 smithi179 ceph-mon[37041]: pgmap v338: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 258 KiB/s wr, 36 op/s 2024-06-03T01:21:12.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:12 smithi107 ceph-mon[36622]: pgmap v338: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 258 KiB/s wr, 36 op/s 2024-06-03T01:21:13.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 240480 2.03 MB/sec execute 440 sec latency 363.531 ms 2024-06-03T01:21:14.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 241566 2.04 MB/sec execute 441 sec latency 15.909 ms 2024-06-03T01:21:14.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:14 smithi094 ceph-mon[32119]: pgmap v339: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 950 KiB/s wr, 99 op/s 2024-06-03T01:21:14.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:14 smithi179 ceph-mon[37041]: pgmap v339: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 950 KiB/s wr, 99 op/s 2024-06-03T01:21:14.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:14 smithi107 ceph-mon[36622]: pgmap v339: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 950 KiB/s wr, 99 op/s 2024-06-03T01:21:15.129 INFO:tasks.workunit.client.0.smithi094.stdout: 1 241905 2.04 MB/sec execute 442 sec latency 512.939 ms 2024-06-03T01:21:16.130 INFO:tasks.workunit.client.0.smithi094.stdout: 1 241962 2.03 MB/sec execute 443 sec latency 1439.301 ms 2024-06-03T01:21:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:16 smithi107 ceph-mon[36622]: pgmap v340: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 892 KiB/s wr, 93 op/s 2024-06-03T01:21:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:16 smithi094 ceph-mon[32119]: pgmap v340: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 892 KiB/s wr, 93 op/s 2024-06-03T01:21:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:16 smithi179 ceph-mon[37041]: pgmap v340: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 892 KiB/s wr, 93 op/s 2024-06-03T01:21:17.130 INFO:tasks.workunit.client.0.smithi094.stdout: 1 242829 2.04 MB/sec execute 444 sec latency 19.942 ms 2024-06-03T01:21:18.130 INFO:tasks.workunit.client.0.smithi094.stdout: 1 244012 2.05 MB/sec execute 445 sec latency 74.385 ms 2024-06-03T01:21:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:18 smithi107 ceph-mon[36622]: pgmap v341: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-06-03T01:21:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:18 smithi094 ceph-mon[32119]: pgmap v341: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-06-03T01:21:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:18 smithi179 ceph-mon[37041]: pgmap v341: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-06-03T01:21:19.130 INFO:tasks.workunit.client.0.smithi094.stdout: 1 245150 2.06 MB/sec execute 446 sec latency 18.697 ms 2024-06-03T01:21:20.130 INFO:tasks.workunit.client.0.smithi094.stdout: 1 245577 2.06 MB/sec execute 447 sec latency 361.089 ms 2024-06-03T01:21:20.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:20 smithi107 ceph-mon[36622]: pgmap v342: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-06-03T01:21:20.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:20 smithi094 ceph-mon[32119]: pgmap v342: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-06-03T01:21:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:20 smithi179 ceph-mon[37041]: pgmap v342: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-06-03T01:21:21.130 INFO:tasks.workunit.client.0.smithi094.stdout: 1 245670 2.05 MB/sec execute 448 sec latency 1283.767 ms 2024-06-03T01:21:22.130 INFO:tasks.workunit.client.0.smithi094.stdout: 1 246478 2.06 MB/sec execute 449 sec latency 222.505 ms 2024-06-03T01:21:22.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:22 smithi107 ceph-mon[36622]: pgmap v343: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 985 KiB/s wr, 105 op/s 2024-06-03T01:21:22.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:22 smithi094 ceph-mon[32119]: pgmap v343: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 985 KiB/s wr, 105 op/s 2024-06-03T01:21:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:22 smithi179 ceph-mon[37041]: pgmap v343: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 985 KiB/s wr, 105 op/s 2024-06-03T01:21:23.131 INFO:tasks.workunit.client.0.smithi094.stdout: 1 247387 2.06 MB/sec execute 450 sec latency 477.224 ms 2024-06-03T01:21:24.131 INFO:tasks.workunit.client.0.smithi094.stdout: 1 248558 2.08 MB/sec execute 451 sec latency 26.805 ms 2024-06-03T01:21:24.406 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:24 smithi179 ceph-mon[37041]: pgmap v344: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-03T01:21:24.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:24 smithi107 ceph-mon[36622]: pgmap v344: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-03T01:21:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:24 smithi094 ceph-mon[32119]: pgmap v344: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-03T01:21:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 -- 172.21.15.94:0/1528239619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 msgr2=0x7fbc5406a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 --2- 172.21.15.94:0/1528239619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 0x7fbc5406a630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009930 tx=0x7fbc4002a420 comp rx=0 tx=0).stop 2024-06-03T01:21:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 -- 172.21.15.94:0/1528239619 shutdown_connections 2024-06-03T01:21:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 --2- 172.21.15.94:0/1528239619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc5406ab70 0x7fbc5410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 --2- 172.21.15.94:0/1528239619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 0x7fbc5406a630 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 --2- 172.21.15.94:0/1528239619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 0x7fbc54107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 -- 172.21.15.94:0/1528239619 >> 172.21.15.94:0/1528239619 conn(0x7fbc54077710 msgr2=0x7fbc54077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 -- 172.21.15.94:0/1528239619 shutdown_connections 2024-06-03T01:21:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.085+0000 7fbc5c000640 1 -- 172.21.15.94:0/1528239619 wait complete. 2024-06-03T01:21:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.086+0000 7fbc5c000640 1 Processor -- start 2024-06-03T01:21:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5c000640 1 -- start start 2024-06-03T01:21:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5c000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 0x7fbc5419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5c000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc5406ab70 0x7fbc5419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5c000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 0x7fbc5419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5c000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54111e20 con 0x7fbc5406ab70 2024-06-03T01:21:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5c000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54111ca0 con 0x7fbc5406a1b0 2024-06-03T01:21:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5c000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54111fa0 con 0x7fbc54107a70 2024-06-03T01:21:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 0x7fbc5419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc58c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc5406ab70 0x7fbc5419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc59600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 0x7fbc5419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 0x7fbc5419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:50400/0 (socket says 172.21.15.94:50400) 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.087+0000 7fbc5a000640 1 -- 172.21.15.94:0/3041851686 learned_addr learned my addr 172.21.15.94:0/3041851686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc5a000640 1 -- 172.21.15.94:0/3041851686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 msgr2=0x7fbc5419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc5a000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 0x7fbc5419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc5a000640 1 -- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc5406ab70 msgr2=0x7fbc5419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc5a000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc5406ab70 0x7fbc5419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc5a000640 1 -- 172.21.15.94:0/3041851686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5419f350 con 0x7fbc54107a70 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc59600640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 0x7fbc5419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc5a000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 0x7fbc5419ec20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00e970 tx=0x7fbc4c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc52000640 1 -- 172.21.15.94:0/3041851686 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c00ccc0 con 0x7fbc54107a70 2024-06-03T01:21:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5419f640 con 0x7fbc54107a70 2024-06-03T01:21:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc52000640 1 -- 172.21.15.94:0/3041851686 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4c00ce60 con 0x7fbc54107a70 2024-06-03T01:21:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.088+0000 7fbc52000640 1 -- 172.21.15.94:0/3041851686 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c010430 con 0x7fbc54107a70 2024-06-03T01:21:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.089+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc541a6ed0 con 0x7fbc54107a70 2024-06-03T01:21:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 Processor -- start 2024-06-03T01:21:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc52000640 1 -- 172.21.15.94:0/3041851686 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbc4c005000 con 0x7fbc54107a70 2024-06-03T01:21:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 -- start start 2024-06-03T01:21:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d7125b0 0x55ec2d710b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d711040 0x55ec2d7114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ec2d675420 con 0x55ec2d7125b0 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ec2d61cc50 con 0x55ec2d711040 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.092+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ec2d6713e0 con 0x55ec2d71bdc0 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc58c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d7125b0 0x55ec2d710b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc58c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d7125b0 0x55ec2d710b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60494/0 (socket says 172.21.15.94:60494) 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc58c00640 1 -- 172.21.15.94:0/4290133151 learned_addr learned my addr 172.21.15.94:0/4290133151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc59600640 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d711040 0x55ec2d7114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc58c00640 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc59600640 1 -- 172.21.15.94:0/4290133151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 msgr2=0x55ec2d71e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc59600640 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc59600640 1 -- 172.21.15.94:0/4290133151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d7125b0 msgr2=0x55ec2d710b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc59600640 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d7125b0 0x55ec2d710b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc59600640 1 -- 172.21.15.94:0/4290133151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec2d389210 con 0x55ec2d711040 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc58c00640 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.093+0000 7fbc59600640 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d711040 0x55ec2d7114c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbc480117e0 tx=0x7fbc48011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e67c34d42f03247 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc52000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbc3c0a2da0 0x7fbc3c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc46600640 1 -- 172.21.15.94:0/4290133151 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48003c30 con 0x55ec2d711040 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc46600640 1 -- 172.21.15.94:0/4290133151 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4800fe80 con 0x55ec2d711040 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc46600640 1 -- 172.21.15.94:0/4290133151 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc480193e0 con 0x55ec2d711040 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc52000640 1 -- 172.21.15.94:0/3041851686 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7fbc4c0cfbe0 con 0x7fbc54107a70 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc58c00640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbc3c0a2da0 0x7fbc3c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc5ee13740 1 -- 172.21.15.94:0/4290133151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d711040 msgr2=0x55ec2d7114c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d711040 0x55ec2d7114c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbc480117e0 tx=0x7fbc48011ca0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc5ee13740 1 -- 172.21.15.94:0/4290133151 shutdown_connections 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d711040 0x55ec2d7114c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/4290133151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d7125b0 0x55ec2d710b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.094+0000 7fbc5ee13740 1 -- 172.21.15.94:0/4290133151 >> 172.21.15.94:0/4290133151 conn(0x55ec2d6d9f80 msgr2=0x55ec2d70d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.095+0000 7fbc58c00640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbc3c0a2da0 0x7fbc3c0a5260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ec2d70e7c0 tx=0x7fbc40041040 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.095+0000 7fbc5ee13740 1 -- 172.21.15.94:0/4290133151 shutdown_connections 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.095+0000 7fbc5ee13740 1 -- 172.21.15.94:0/4290133151 wait complete. 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.095+0000 7fbc5ee13740 1 -- start start 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.095+0000 7fbc5ee13740 1 Processor -- start 2024-06-03T01:21:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d718dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d719310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d719850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ec2d71eb90 con 0x55ec2d711040 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ec2d6300b0 con 0x55ec2d7125b0 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ec2d71ed10 con 0x55ec2d71bdc0 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5ee13740 10 client.0 _ll_register_callbacks cb 0x55ec2d61bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d718dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d718dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60498/0 (socket says 172.21.15.94:60498) 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 -- 172.21.15.94:0/1045541731 learned_addr learned my addr 172.21.15.94:0/1045541731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc58c00640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d719310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc5a000640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d719850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 -- 172.21.15.94:0/1045541731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 msgr2=0x55ec2d719850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d719850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 -- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 msgr2=0x55ec2d719310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d719310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc59600640 1 -- 172.21.15.94:0/1045541731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec2d455620 con 0x55ec2d711040 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.096+0000 7fbc58c00640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d719310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc59600640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d718dd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc48005c10 tx=0x7fbc48005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2666c150aea0bd75 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc3aa00640 1 -- 172.21.15.94:0/1045541731 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48027020 con 0x55ec2d711040 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ec2d682b50 con 0x55ec2d711040 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc3aa00640 1 -- 172.21.15.94:0/1045541731 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7fbc48004260 con 0x55ec2d711040 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc3aa00640 1 -- 172.21.15.94:0/1045541731 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbc48032050 con 0x55ec2d711040 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc5ee13740 10 client.16065 fetch_fsmap learned FSMap version 54 2024-06-03T01:21:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.097+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ec2d72af00 con 0x55ec2d711040 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc3aa00640 1 -- 172.21.15.94:0/1045541731 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ec2d682b50 con 0x55ec2d711040 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc3aa00640 1 -- 172.21.15.94:0/1045541731 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fbc48018070 con 0x55ec2d711040 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc5ee13740 10 client.16065 fetch_fsmap finished waiting for FSMap version 54 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc5ee13740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc5ee13740 20 client.16065 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d72c300 0x55ec2d72e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc5ee13740 4 client.16065 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec2d67cea0 con 0x55ec2d72c300 2024-06-03T01:21:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.098+0000 7fbc58c00640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d72c300 0x55ec2d72e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.099+0000 7fbc58c00640 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d72c300 0x55ec2d72e6f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a50a52edfd9366b server_cookie=edba9e603700adbc in_seq=0 out_seq=0 2024-06-03T01:21:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.099+0000 7fbc3aa00640 10 client.16065 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.099+0000 7fbc3aa00640 1 -- 172.21.15.94:0/1045541731 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec2d67cea0 con 0x55ec2d72c300 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.099+0000 7fbc3aa00640 10 client.16065 handle_command_reply: tid=0 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.099+0000 7fbc5ee13740 1 client.16065 shutdown 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 msgr2=0x55ec2d718dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d718dd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbc48005c10 tx=0x7fbc48005ad0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 shutdown_connections 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d719850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d72c300 0x55ec2d72e6f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d719310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/1045541731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d718dd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 >> 172.21.15.94:0/1045541731 conn(0x55ec2d6d9f80 msgr2=0x55ec2d7134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc3aa00640 0 client.16065 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 shutdown_connections 2024-06-03T01:21:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 1 -- 172.21.15.94:0/1045541731 wait complete. 2024-06-03T01:21:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.100+0000 7fbc5ee13740 20 client.16065 trim_cache size 0 max 16384 2024-06-03T01:21:25.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.116+0000 7fbc5ee13740 1 Processor -- start 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5ee13740 1 -- start start 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d70d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541b0ff0 con 0x55ec2d711040 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541b2c50 con 0x55ec2d7125b0 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541b16c0 con 0x55ec2d71bdc0 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc59600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d70d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc58c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc59600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d70d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60508/0 (socket says 172.21.15.94:60508) 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc59600640 1 -- 172.21.15.94:0/3646498713 learned_addr learned my addr 172.21.15.94:0/3646498713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.117+0000 7fbc5a000640 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc59600640 1 -- 172.21.15.94:0/3646498713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 msgr2=0x55ec2d71d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc59600640 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc59600640 1 -- 172.21.15.94:0/3646498713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 msgr2=0x55ec2d71d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc59600640 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc59600640 1 -- 172.21.15.94:0/3646498713 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec2d72ec50 con 0x55ec2d711040 2024-06-03T01:21:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc5a000640 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:21:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc59600640 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d70d780 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbc48000f80 tx=0x7fbc480021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb37f76241c8bdcc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc45c00640 1 -- 172.21.15.94:0/3646498713 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48027040 con 0x55ec2d711040 2024-06-03T01:21:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc45c00640 1 -- 172.21.15.94:0/3646498713 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc48004600 con 0x55ec2d711040 2024-06-03T01:21:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.118+0000 7fbc45c00640 1 -- 172.21.15.94:0/3646498713 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4801a860 con 0x55ec2d711040 2024-06-03T01:21:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 -- 172.21.15.94:0/3646498713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 msgr2=0x55ec2d70d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d70d780 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbc48000f80 tx=0x7fbc480021d0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 -- 172.21.15.94:0/3646498713 shutdown_connections 2024-06-03T01:21:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/3646498713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d711040 0x55ec2d70d780 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 -- 172.21.15.94:0/3646498713 >> 172.21.15.94:0/3646498713 conn(0x55ec2d6d9f80 msgr2=0x55ec2d70d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 -- 172.21.15.94:0/3646498713 shutdown_connections 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 -- 172.21.15.94:0/3646498713 wait complete. 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.119+0000 7fbc5ee13740 1 -- start start 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 1 Processor -- start 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d711040 0x55ec2d71c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541b24f0 con 0x55ec2d71bdc0 2024-06-03T01:21:25.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541b1e20 con 0x55ec2d7125b0 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541aeff0 con 0x55ec2d711040 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5ee13740 10 client.0 _ll_register_callbacks cb 0x7fbc480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60524/0 (socket says 172.21.15.94:60524) 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc5a000640 1 -- 172.21.15.94:0/2929351911 learned_addr learned my addr 172.21.15.94:0/2929351911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc58c00640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.120+0000 7fbc59600640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d711040 0x55ec2d71c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5a000640 1 -- 172.21.15.94:0/2929351911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d711040 msgr2=0x55ec2d71c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5a000640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d711040 0x55ec2d71c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5a000640 1 -- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 msgr2=0x55ec2d71c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5a000640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5a000640 1 -- 172.21.15.94:0/2929351911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec2d723120 con 0x55ec2d71bdc0 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc58c00640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5a000640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71cc20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c09bc70 tx=0x7fbc4c09bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49d97997a24b90a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc3aa00640 1 -- 172.21.15.94:0/2929351911 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0052f0 con 0x55ec2d71bdc0 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc3aa00640 1 -- 172.21.15.94:0/2929351911 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7fbc4c0a44c0 con 0x55ec2d71bdc0 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc48004600 con 0x55ec2d71bdc0 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc3aa00640 1 -- 172.21.15.94:0/2929351911 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c081020 con 0x55ec2d71bdc0 2024-06-03T01:21:25.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5ee13740 10 client.16071 fetch_fsmap learned FSMap version 54 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.121+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ec2d7243e0 con 0x55ec2d71bdc0 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc3aa00640 1 -- 172.21.15.94:0/2929351911 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc48004600 con 0x55ec2d71bdc0 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc3aa00640 1 -- 172.21.15.94:0/2929351911 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 54) ==== 8386+0+0 (secure 0 0 0) 0x7fbc4c09aba0 con 0x55ec2d71bdc0 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc5ee13740 10 client.16071 fetch_fsmap finished waiting for FSMap version 54 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc5ee13740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc5ee13740 20 client.16071 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d79f660 0x55ec2d7a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc5ee13740 4 client.16071 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec2d5fa7a0 con 0x55ec2d79f660 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.122+0000 7fbc59600640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d79f660 0x55ec2d7a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.123+0000 7fbc59600640 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d79f660 0x55ec2d7a1a50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd98b5fe735c241 server_cookie=e613f86db9e9964a in_seq=0 out_seq=0 2024-06-03T01:21:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.123+0000 7fbc3aa00640 10 client.16071 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:25.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249052 2.08 MB/sec execute 452 sec latency 56.539 ms 2024-06-03T01:21:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.142+0000 7fbc3aa00640 1 -- 172.21.15.94:0/2929351911 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ec2d5fa7a0 con 0x55ec2d79f660 2024-06-03T01:21:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.142+0000 7fbc3aa00640 10 client.16071 handle_command_reply: tid=0 2024-06-03T01:21:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.142+0000 7fbc5ee13740 1 client.16071 shutdown 2024-06-03T01:21:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.142+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d71bdc0 msgr2=0x55ec2d71cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.142+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71cc20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c09bc70 tx=0x7fbc4c09bda0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 shutdown_connections 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ec2d71bdc0 0x55ec2d71cc20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55ec2d79f660 0x55ec2d7a1a50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec2d7125b0 0x55ec2d71c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 --2- 172.21.15.94:0/2929351911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec2d711040 0x55ec2d71c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc3aa00640 0 client.16071 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 >> 172.21.15.94:0/2929351911 conn(0x55ec2d6d9f80 msgr2=0x55ec2d70bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 shutdown_connections 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 1 -- 172.21.15.94:0/2929351911 wait complete. 2024-06-03T01:21:25.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.143+0000 7fbc5ee13740 20 client.16071 trim_cache size 0 max 16384 2024-06-03T01:21:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:21:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:21:25.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:21:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbc3c0a2da0 msgr2=0x7fbc3c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbc3c0a2da0 0x7fbc3c0a5260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55ec2d70e7c0 tx=0x7fbc40041040 comp rx=0 tx=0).stop 2024-06-03T01:21:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 msgr2=0x7fbc5419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 0x7fbc5419ec20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00e970 tx=0x7fbc4c00ee30 comp rx=0 tx=0).stop 2024-06-03T01:21:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 shutdown_connections 2024-06-03T01:21:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbc3c0a2da0 0x7fbc3c0a5260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc54107a70 0x7fbc5419ec20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc5406ab70 0x7fbc5419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 --2- 172.21.15.94:0/3041851686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc5406a1b0 0x7fbc5419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 >> 172.21.15.94:0/3041851686 conn(0x7fbc54077710 msgr2=0x7fbc5410ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 shutdown_connections 2024-06-03T01:21:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:25.144+0000 7fbc5c000640 1 -- 172.21.15.94:0/3041851686 wait complete. 2024-06-03T01:21:26.131 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249110 2.07 MB/sec execute 453 sec latency 983.617 ms 2024-06-03T01:21:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:26 smithi107 ceph-mon[36622]: pgmap v345: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 766 KiB/s wr, 105 op/s 2024-06-03T01:21:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:26 smithi094 ceph-mon[32119]: pgmap v345: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 766 KiB/s wr, 105 op/s 2024-06-03T01:21:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:26 smithi179 ceph-mon[37041]: pgmap v345: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 766 KiB/s wr, 105 op/s 2024-06-03T01:21:27.131 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249120 2.07 MB/sec execute 454 sec latency 980.774 ms 2024-06-03T01:21:28.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249120 2.06 MB/sec execute 455 sec latency 1980.870 ms 2024-06-03T01:21:28.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:28 smithi107 ceph-mon[36622]: pgmap v346: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-03T01:21:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:28 smithi094 ceph-mon[32119]: pgmap v346: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-03T01:21:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:28 smithi179 ceph-mon[37041]: pgmap v346: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-03T01:21:29.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249120 2.06 MB/sec execute 456 sec latency 2981.010 ms 2024-06-03T01:21:29.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:29 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:29.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:29 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:29.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:29 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:30.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249120 2.05 MB/sec execute 457 sec latency 3981.187 ms 2024-06-03T01:21:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:30 smithi107 ceph-mon[36622]: pgmap v347: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 861 KiB/s wr, 104 op/s 2024-06-03T01:21:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:30 smithi094 ceph-mon[32119]: pgmap v347: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 861 KiB/s wr, 104 op/s 2024-06-03T01:21:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:30 smithi179 ceph-mon[37041]: pgmap v347: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 861 KiB/s wr, 104 op/s 2024-06-03T01:21:31.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249197 2.05 MB/sec execute 458 sec latency 4909.396 ms 2024-06-03T01:21:32.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249732 2.06 MB/sec execute 459 sec latency 488.308 ms 2024-06-03T01:21:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:32 smithi107 ceph-mon[36622]: pgmap v348: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 861 KiB/s wr, 104 op/s 2024-06-03T01:21:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:32 smithi094 ceph-mon[32119]: pgmap v348: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 861 KiB/s wr, 104 op/s 2024-06-03T01:21:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:32 smithi179 ceph-mon[37041]: pgmap v348: 129 pgs: 129 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 861 KiB/s wr, 104 op/s 2024-06-03T01:21:33.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249825 2.05 MB/sec execute 460 sec latency 735.008 ms 2024-06-03T01:21:34.132 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249825 2.05 MB/sec execute 461 sec latency 1611.410 ms 2024-06-03T01:21:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:34 smithi107 ceph-mon[36622]: pgmap v349: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-06-03T01:21:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:34 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:34 smithi094 ceph-mon[32119]: pgmap v349: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-06-03T01:21:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:34 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:34 smithi179 ceph-mon[37041]: pgmap v349: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-06-03T01:21:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:34 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:21:35.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249825 2.04 MB/sec execute 462 sec latency 2611.518 ms 2024-06-03T01:21:36.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249825 2.04 MB/sec execute 463 sec latency 3611.707 ms 2024-06-03T01:21:36.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:36 smithi107 ceph-mon[36622]: pgmap v350: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 839 KiB/s wr, 74 op/s 2024-06-03T01:21:36.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:36 smithi094 ceph-mon[32119]: pgmap v350: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 839 KiB/s wr, 74 op/s 2024-06-03T01:21:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:36 smithi179 ceph-mon[37041]: pgmap v350: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 839 KiB/s wr, 74 op/s 2024-06-03T01:21:37.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249825 2.03 MB/sec execute 464 sec latency 4611.803 ms 2024-06-03T01:21:38.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 249835 2.03 MB/sec execute 465 sec latency 5606.824 ms 2024-06-03T01:21:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.295+0000 7f3e33400640 1 -- 172.21.15.94:0/3628037777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c102c30 msgr2=0x7f3e2c103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.295+0000 7f3e33400640 1 --2- 172.21.15.94:0/3628037777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c102c30 0x7f3e2c103010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009900 tx=0x7f3e1c02a480 comp rx=0 tx=0).stop 2024-06-03T01:21:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.296+0000 7f3e33400640 1 -- 172.21.15.94:0/3628037777 shutdown_connections 2024-06-03T01:21:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.296+0000 7f3e33400640 1 --2- 172.21.15.94:0/3628037777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c10c000 0x7f3e2c10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.296+0000 7f3e33400640 1 --2- 172.21.15.94:0/3628037777 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2c103550 0x7f3e2c10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.296+0000 7f3e33400640 1 --2- 172.21.15.94:0/3628037777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c102c30 0x7f3e2c103010 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.296+0000 7f3e33400640 1 -- 172.21.15.94:0/3628037777 >> 172.21.15.94:0/3628037777 conn(0x7f3e2c0779b0 msgr2=0x7f3e2c077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.296+0000 7f3e33400640 1 -- 172.21.15.94:0/3628037777 shutdown_connections 2024-06-03T01:21:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.296+0000 7f3e33400640 1 -- 172.21.15.94:0/3628037777 wait complete. 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 Processor -- start 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 -- start start 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c102c30 0x7f3e2c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2c103550 0x7f3e2c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c10c000 0x7f3e2c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c111d90 con 0x7f3e2c103550 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c111c10 con 0x7f3e2c102c30 2024-06-03T01:21:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.297+0000 7f3e33400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c111f10 con 0x7f3e2c10c000 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c102c30 0x7f3e2c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c102c30 0x7f3e2c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56972/0 (socket says 172.21.15.94:56972) 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e31400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c10c000 0x7f3e2c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e30a00640 1 -- 172.21.15.94:0/3013812540 learned_addr learned my addr 172.21.15.94:0/3013812540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e2be00640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2c103550 0x7f3e2c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e31400640 1 -- 172.21.15.94:0/3013812540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c102c30 msgr2=0x7f3e2c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e31400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c102c30 0x7f3e2c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e31400640 1 -- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2c103550 msgr2=0x7f3e2c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e31400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2c103550 0x7f3e2c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e31400640 1 -- 172.21.15.94:0/3013812540 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c19d1e0 con 0x7f3e2c10c000 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e30a00640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c102c30 0x7f3e2c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e2be00640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2c103550 0x7f3e2c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.298+0000 7f3e31400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c10c000 0x7f3e2c19cab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e24005fd0 tx=0x7f3e2400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.299+0000 7f3e29600640 1 -- 172.21.15.94:0/3013812540 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400cdf0 con 0x7f3e2c10c000 2024-06-03T01:21:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.299+0000 7f3e29600640 1 -- 172.21.15.94:0/3013812540 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e24005020 con 0x7f3e2c10c000 2024-06-03T01:21:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.299+0000 7f3e29600640 1 -- 172.21.15.94:0/3013812540 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240106d0 con 0x7f3e2c10c000 2024-06-03T01:21:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.299+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c19d4d0 con 0x7f3e2c10c000 2024-06-03T01:21:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.299+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c1a4d60 con 0x7f3e2c10c000 2024-06-03T01:21:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.302+0000 7f3e361c1740 1 Processor -- start 2024-06-03T01:21:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.303+0000 7f3e29600640 1 -- 172.21.15.94:0/3013812540 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3e240054d0 con 0x7f3e2c10c000 2024-06-03T01:21:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.303+0000 7f3e361c1740 1 -- start start 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.303+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f8913f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f89215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e361c1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5573f8874420 con 0x5573f891f1d0 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e361c1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5573f881bc50 con 0x5573f89159c0 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e361c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5573f88703e0 con 0x5573f8914450 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:52306/0 (socket says 172.21.15.94:52306) 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 -- 172.21.15.94:0/590217327 learned_addr learned my addr 172.21.15.94:0/590217327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e2be00640 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f8913f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e2be00640 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f89215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 -- 172.21.15.94:0/590217327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 msgr2=0x5573f8913f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f8913f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 -- 172.21.15.94:0/590217327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 msgr2=0x5573f89215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f89215c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.304+0000 7f3e30a00640 1 -- 172.21.15.94:0/590217327 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f8588210 con 0x5573f8914450 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.305+0000 7f3e29600640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3e100a2da0 0x7f3e100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.305+0000 7f3e30a00640 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89148d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c005dd0 tx=0x7f3e1c005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2976a06d3062d456 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.305+0000 7f3e29600640 1 -- 172.21.15.94:0/3013812540 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7f3e240cfe10 con 0x7f3e2c10c000 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.305+0000 7f3e2be00640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3e100a2da0 0x7f3e100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.305+0000 7f3e21c00640 1 -- 172.21.15.94:0/590217327 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c035040 con 0x5573f8914450 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.305+0000 7f3e21c00640 1 -- 172.21.15.94:0/590217327 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e1c0319f0 con 0x5573f8914450 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.305+0000 7f3e21c00640 1 -- 172.21.15.94:0/590217327 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c0049c0 con 0x5573f8914450 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 -- 172.21.15.94:0/590217327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 msgr2=0x5573f89148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e2be00640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3e100a2da0 0x7f3e100a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5573f8911bd0 tx=0x7f3e14004730 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89148d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c005dd0 tx=0x7f3e1c005e00 comp rx=0 tx=0).stop 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 -- 172.21.15.94:0/590217327 shutdown_connections 2024-06-03T01:21:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f89215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89148d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 --2- 172.21.15.94:0/590217327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f8913f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 -- 172.21.15.94:0/590217327 >> 172.21.15.94:0/590217327 conn(0x5573f88d8f80 msgr2=0x5573f8910a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 -- 172.21.15.94:0/590217327 shutdown_connections 2024-06-03T01:21:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.306+0000 7f3e361c1740 1 -- 172.21.15.94:0/590217327 wait complete. 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 -- start start 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 Processor -- start 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f8914450 0x5573f891c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f891c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f891f1d0 0x5573f891ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5573f8921f40 con 0x5573f8914450 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5573f882f0b0 con 0x5573f89159c0 2024-06-03T01:21:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.307+0000 7f3e361c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5573f89220c0 con 0x5573f891f1d0 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e361c1740 10 client.0 _ll_register_callbacks cb 0x5573f881af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f891c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f8914450 0x5573f891c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f891c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56986/0 (socket says 172.21.15.94:56986) 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e31400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f891f1d0 0x5573f891ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e2be00640 1 -- 172.21.15.94:0/2145968280 learned_addr learned my addr 172.21.15.94:0/2145968280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e31400640 1 -- 172.21.15.94:0/2145968280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 msgr2=0x5573f891c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e31400640 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f891c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e31400640 1 -- 172.21.15.94:0/2145968280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f8914450 msgr2=0x5573f891c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e31400640 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f8914450 0x5573f891c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e31400640 1 -- 172.21.15.94:0/2145968280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f8654620 con 0x5573f891f1d0 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e31400640 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f891f1d0 0x5573f891ccb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e240cd140 tx=0x7f3e2409bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ece338aebfd759e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.308+0000 7f3e1a000640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e1a000640 1 -- 172.21.15.94:0/2145968280 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e240a46f0 con 0x5573f891f1d0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e1a000640 1 -- 172.21.15.94:0/2145968280 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7f3e240a4890 con 0x5573f891f1d0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1c0319f0 con 0x5573f891f1d0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e1a000640 1 -- 172.21.15.94:0/2145968280 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3e240bab50 con 0x5573f891f1d0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 10 client.26206 fetch_fsmap learned FSMap version 56 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5573f892e360 con 0x5573f891f1d0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e1a000640 1 -- 172.21.15.94:0/2145968280 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e1c0319f0 con 0x5573f891f1d0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e1a000640 1 -- 172.21.15.94:0/2145968280 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f3e2409b4f0 con 0x5573f891f1d0 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 10 client.26206 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 10 client.26206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 20 client.26206 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f892f890 0x5573f8931ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 4 client.26206 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:21:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.309+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573f8881b50 con 0x5573f892f890 2024-06-03T01:21:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.310+0000 7f3e30a00640 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f892f890 0x5573f8931ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.310+0000 7f3e30a00640 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f892f890 0x5573f8931ce0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf32d708e8ec4ac server_cookie=99e8ff0bb7df46e3 in_seq=0 out_seq=0 2024-06-03T01:21:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.310+0000 7f3e1a000640 10 client.26206 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e1a000640 1 -- 172.21.15.94:0/2145968280 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5573f8881b50 con 0x5573f892f890 2024-06-03T01:21:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e1a000640 10 client.26206 handle_command_reply: tid=0 2024-06-03T01:21:38.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 client.26206 shutdown 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f891f1d0 msgr2=0x5573f891ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f891f1d0 0x5573f891ccb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e240cd140 tx=0x7f3e2409bf10 comp rx=0 tx=0).stop 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 shutdown_connections 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f891f1d0 0x5573f891ccb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f892f890 0x5573f8931ce0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f89159c0 0x5573f891c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2145968280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f8914450 0x5573f891c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 >> 172.21.15.94:0/2145968280 conn(0x5573f88d8f80 msgr2=0x5573f890b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.311+0000 7f3e1a000640 0 client.26206 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.312+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 shutdown_connections 2024-06-03T01:21:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.312+0000 7f3e361c1740 1 -- 172.21.15.94:0/2145968280 wait complete. 2024-06-03T01:21:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.312+0000 7f3e361c1740 20 client.26206 trim_cache size 0 max 16384 2024-06-03T01:21:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 Processor -- start 2024-06-03T01:21:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 -- start start 2024-06-03T01:21:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f8914450 0x5573f890b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f89159c0 0x5573f89205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f8920b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1acc20 con 0x5573f891f1d0 2024-06-03T01:21:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ae8b0 con 0x5573f8914450 2024-06-03T01:21:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.330+0000 7f3e361c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ad320 con 0x5573f89159c0 2024-06-03T01:21:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f89159c0 0x5573f89205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f8914450 0x5573f890b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f89159c0 0x5573f89205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:52320/0 (socket says 172.21.15.94:52320) 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 -- 172.21.15.94:0/2926816874 learned_addr learned my addr 172.21.15.94:0/2926816874 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f8914450 0x5573f890b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:56998/0 (socket says 172.21.15.94:56998) 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e31400640 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f8920b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 -- 172.21.15.94:0/2926816874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f8914450 msgr2=0x5573f890b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f8914450 0x5573f890b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 -- 172.21.15.94:0/2926816874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 msgr2=0x5573f8920b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f8920b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e2be00640 1 -- 172.21.15.94:0/2926816874 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f891a780 con 0x5573f89159c0 2024-06-03T01:21:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.331+0000 7f3e30a00640 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f8914450 0x5573f890b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.332+0000 7f3e2be00640 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f89159c0 0x5573f89205c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400f910 tx=0x7f3e1401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=774423b257f86bae server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.332+0000 7f3e21200640 1 -- 172.21.15.94:0/2926816874 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14004050 con 0x5573f89159c0 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.332+0000 7f3e21200640 1 -- 172.21.15.94:0/2926816874 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14020e20 con 0x5573f89159c0 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.332+0000 7f3e21200640 1 -- 172.21.15.94:0/2926816874 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14005490 con 0x5573f89159c0 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 -- 172.21.15.94:0/2926816874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f89159c0 msgr2=0x5573f89205c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f89159c0 0x5573f89205c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400f910 tx=0x7f3e1401c000 comp rx=0 tx=0).stop 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 -- 172.21.15.94:0/2926816874 shutdown_connections 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f891f1d0 0x5573f8920b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f89159c0 0x5573f89205c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 --2- 172.21.15.94:0/2926816874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f8914450 0x5573f890b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 -- 172.21.15.94:0/2926816874 >> 172.21.15.94:0/2926816874 conn(0x5573f88d8f80 msgr2=0x5573f890af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 -- 172.21.15.94:0/2926816874 shutdown_connections 2024-06-03T01:21:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 -- 172.21.15.94:0/2926816874 wait complete. 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 -- start start 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.333+0000 7f3e361c1740 1 Processor -- start 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f89159c0 0x5573f8936d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e361c1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f891f1d0 0x5573f89372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e361c1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ae150 con 0x5573f89159c0 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e361c1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ada80 con 0x5573f891f1d0 2024-06-03T01:21:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e361c1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1aac20 con 0x5573f8914450 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e361c1740 10 client.0 _ll_register_callbacks cb 0x7f3e240981b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f89159c0 0x5573f8936d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e31400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f891f1d0 0x5573f89372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e31400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f891f1d0 0x5573f89372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:57006/0 (socket says 172.21.15.94:57006) 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f89159c0 0x5573f8936d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35002/0 (socket says 172.21.15.94:35002) 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e31400640 1 -- 172.21.15.94:0/1756242197 learned_addr learned my addr 172.21.15.94:0/1756242197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.334+0000 7f3e30a00640 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e31400640 1 -- 172.21.15.94:0/1756242197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 msgr2=0x5573f89278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e31400640 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e31400640 1 -- 172.21.15.94:0/1756242197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f89159c0 msgr2=0x5573f8936d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e31400640 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f89159c0 0x5573f8936d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e31400640 1 -- 172.21.15.94:0/1756242197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f89378b0 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e31400640 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f891f1d0 0x5573f89372c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3e2409c7c0 tx=0x7f3e2400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab1311803592e9f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e1a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e1a000640 1 -- 172.21.15.94:0/1756242197 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2407f030 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e1a000640 1 -- 172.21.15.94:0/1756242197 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7f3e240a4900 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e14020e20 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e1a000640 1 -- 172.21.15.94:0/1756242197 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3e240a9050 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e361c1740 10 client.26204 fetch_fsmap learned FSMap version 56 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.335+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5573f8926230 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e1a000640 1 -- 172.21.15.94:0/1756242197 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14020e20 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e1a000640 1 -- 172.21.15.94:0/1756242197 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f3e2409b360 con 0x5573f891f1d0 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e361c1740 10 client.26204 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:21:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e361c1740 10 client.26204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e361c1740 20 client.26204 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e361c1740 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f899ab40 0x5573f899cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e361c1740 4 client.26204 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:21:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573f887bea0 con 0x5573f899ab40 2024-06-03T01:21:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.336+0000 7f3e30a00640 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f899ab40 0x5573f899cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.337+0000 7f3e30a00640 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f899ab40 0x5573f899cf60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2dbb1ba1f982d server_cookie=da3eb85bae7bcef3 in_seq=0 out_seq=0 2024-06-03T01:21:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.337+0000 7f3e1a000640 10 client.26204 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e1a000640 1 -- 172.21.15.94:0/1756242197 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5573f887bea0 con 0x5573f899ab40 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e1a000640 10 client.26204 handle_command_reply: tid=0 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 client.26204 shutdown 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f891f1d0 msgr2=0x5573f89372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f891f1d0 0x5573f89372c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3e2409c7c0 tx=0x7f3e2400cf90 comp rx=0 tx=0).stop 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 shutdown_connections 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5573f899ab40 0x5573f899cf60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573f891f1d0 0x5573f89372c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f89159c0 0x5573f8936d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 --2- 172.21.15.94:0/1756242197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5573f8914450 0x5573f89278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e1a000640 0 client.26204 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 >> 172.21.15.94:0/1756242197 conn(0x5573f88d8f80 msgr2=0x5573f890b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 shutdown_connections 2024-06-03T01:21:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.358+0000 7f3e361c1740 1 -- 172.21.15.94:0/1756242197 wait complete. 2024-06-03T01:21:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.359+0000 7f3e361c1740 20 client.26204 trim_cache size 0 max 16384 2024-06-03T01:21:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:21:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:21:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.359+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3e100a2da0 msgr2=0x7f3e100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.359+0000 7f3e33400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3e100a2da0 0x7f3e100a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5573f8911bd0 tx=0x7f3e14004730 comp rx=0 tx=0).stop 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.359+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c10c000 msgr2=0x7f3e2c19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.359+0000 7f3e33400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c10c000 0x7f3e2c19cab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e24005fd0 tx=0x7f3e2400ef10 comp rx=0 tx=0).stop 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 shutdown_connections 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3e100a2da0 0x7f3e100a5260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3e2c10c000 0x7f3e2c19cab0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e2c103550 0x7f3e2c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 --2- 172.21.15.94:0/3013812540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e2c102c30 0x7f3e2c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 >> 172.21.15.94:0/3013812540 conn(0x7f3e2c0779b0 msgr2=0x7f3e2c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 shutdown_connections 2024-06-03T01:21:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:38.360+0000 7f3e33400640 1 -- 172.21.15.94:0/3013812540 wait complete. 2024-06-03T01:21:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:38 smithi107 ceph-mon[36622]: pgmap v351: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 877 KiB/s wr, 88 op/s 2024-06-03T01:21:38.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:38 smithi094 ceph-mon[32119]: pgmap v351: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 877 KiB/s wr, 88 op/s 2024-06-03T01:21:38.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:38 smithi179 ceph-mon[37041]: pgmap v351: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 877 KiB/s wr, 88 op/s 2024-06-03T01:21:39.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 250021 2.03 MB/sec execute 466 sec latency 796.226 ms 2024-06-03T01:21:40.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 250021 2.02 MB/sec execute 467 sec latency 1796.319 ms 2024-06-03T01:21:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:40 smithi107 ceph-mon[36622]: pgmap v352: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 369 KiB/s wr, 38 op/s 2024-06-03T01:21:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:40 smithi094 ceph-mon[32119]: pgmap v352: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 369 KiB/s wr, 38 op/s 2024-06-03T01:21:40.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:40 smithi179 ceph-mon[37041]: pgmap v352: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 369 KiB/s wr, 38 op/s 2024-06-03T01:21:41.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 250021 2.02 MB/sec execute 468 sec latency 2796.454 ms 2024-06-03T01:21:42.133 INFO:tasks.workunit.client.0.smithi094.stdout: 1 250021 2.01 MB/sec execute 469 sec latency 3796.599 ms 2024-06-03T01:21:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:42 smithi107 ceph-mon[36622]: pgmap v353: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 369 KiB/s wr, 38 op/s 2024-06-03T01:21:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:42 smithi094 ceph-mon[32119]: pgmap v353: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 369 KiB/s wr, 38 op/s 2024-06-03T01:21:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:42 smithi179 ceph-mon[37041]: pgmap v353: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 369 KiB/s wr, 38 op/s 2024-06-03T01:21:43.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 250899 2.01 MB/sec execute 470 sec latency 4051.810 ms 2024-06-03T01:21:44.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252103 2.03 MB/sec execute 471 sec latency 30.197 ms 2024-06-03T01:21:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:44 smithi107 ceph-mon[36622]: pgmap v354: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 726 KiB/s wr, 58 op/s 2024-06-03T01:21:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:44 smithi094 ceph-mon[32119]: pgmap v354: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 726 KiB/s wr, 58 op/s 2024-06-03T01:21:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:44 smithi179 ceph-mon[37041]: pgmap v354: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 726 KiB/s wr, 58 op/s 2024-06-03T01:21:45.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252546 2.03 MB/sec execute 472 sec latency 178.685 ms 2024-06-03T01:21:46.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252601 2.03 MB/sec execute 473 sec latency 1102.964 ms 2024-06-03T01:21:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:46 smithi107 ceph-mon[36622]: pgmap v355: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 396 KiB/s wr, 33 op/s 2024-06-03T01:21:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:46 smithi094 ceph-mon[32119]: pgmap v355: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 396 KiB/s wr, 33 op/s 2024-06-03T01:21:46.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:46 smithi179 ceph-mon[37041]: pgmap v355: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 396 KiB/s wr, 33 op/s 2024-06-03T01:21:47.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252602 2.02 MB/sec execute 474 sec latency 989.125 ms 2024-06-03T01:21:48.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252602 2.02 MB/sec execute 475 sec latency 1989.240 ms 2024-06-03T01:21:48.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:48 smithi107 ceph-mon[36622]: pgmap v356: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 874 KiB/s wr, 80 op/s 2024-06-03T01:21:48.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:48 smithi094 ceph-mon[32119]: pgmap v356: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 874 KiB/s wr, 80 op/s 2024-06-03T01:21:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:48 smithi179 ceph-mon[37041]: pgmap v356: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 874 KiB/s wr, 80 op/s 2024-06-03T01:21:49.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252602 2.01 MB/sec execute 476 sec latency 2989.338 ms 2024-06-03T01:21:49.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:49 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:21:49.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:49 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:21:49.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:49 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:21:50.134 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252602 2.01 MB/sec execute 477 sec latency 3989.410 ms 2024-06-03T01:21:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:50 smithi107 ceph-mon[36622]: pgmap v357: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 836 KiB/s wr, 66 op/s 2024-06-03T01:21:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:50 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:21:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:50 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:21:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:50 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:21:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:50 smithi094 ceph-mon[32119]: pgmap v357: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 836 KiB/s wr, 66 op/s 2024-06-03T01:21:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:50 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:21:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:50 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:21:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:50 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:21:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:50 smithi179 ceph-mon[37041]: pgmap v357: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 836 KiB/s wr, 66 op/s 2024-06-03T01:21:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:50 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:21:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:50 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:21:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:50 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:21:51.135 INFO:tasks.workunit.client.0.smithi094.stdout: 1 252669 2.00 MB/sec execute 478 sec latency 4917.179 ms 2024-06-03T01:21:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 -- 172.21.15.94:0/3469722367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 msgr2=0x7ff07c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 --2- 172.21.15.94:0/3469722367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 0x7ff07c107ba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009930 tx=0x7ff06c02a420 comp rx=0 tx=0).stop 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 -- 172.21.15.94:0/3469722367 shutdown_connections 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 --2- 172.21.15.94:0/3469722367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 0x7ff07c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 --2- 172.21.15.94:0/3469722367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c1017b0 0x7ff07c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 --2- 172.21.15.94:0/3469722367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 0x7ff07c107ba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 -- 172.21.15.94:0/3469722367 >> 172.21.15.94:0/3469722367 conn(0x7ff07c0fd320 msgr2=0x7ff07c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 -- 172.21.15.94:0/3469722367 shutdown_connections 2024-06-03T01:21:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.507+0000 7ff080e00640 1 -- 172.21.15.94:0/3469722367 wait complete. 2024-06-03T01:21:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.508+0000 7ff080e00640 1 Processor -- start 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.508+0000 7ff080e00640 1 -- start start 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff080e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c1017b0 0x7ff07c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff080e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 0x7ff07c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff080e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 0x7ff07c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff080e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c111b90 con 0x7ff07c1017b0 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff080e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c111a10 con 0x7ff07c102170 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff080e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c111d10 con 0x7ff07c1077c0 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 0x7ff07c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07b400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c1017b0 0x7ff07c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 0x7ff07c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 0x7ff07c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:50544/0 (socket says 172.21.15.94:50544) 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07aa00640 1 -- 172.21.15.94:0/2562727824 learned_addr learned my addr 172.21.15.94:0/2562727824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07aa00640 1 -- 172.21.15.94:0/2562727824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 msgr2=0x7ff07c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07aa00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 0x7ff07c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.509+0000 7ff07aa00640 1 -- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c1017b0 msgr2=0x7ff07c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff07aa00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c1017b0 0x7ff07c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff07aa00640 1 -- 172.21.15.94:0/2562727824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c19f320 con 0x7ff07c102170 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff07be00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 0x7ff07c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff07b400640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c1017b0 0x7ff07c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff07aa00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 0x7ff07c19a870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9f0 tx=0x7ff06800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff073e00640 1 -- 172.21.15.94:0/2562727824 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06800cd80 con 0x7ff07c102170 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff073e00640 1 -- 172.21.15.94:0/2562727824 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff068010910 con 0x7ff07c102170 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c19f610 con 0x7ff07c102170 2024-06-03T01:21:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff073e00640 1 -- 172.21.15.94:0/2562727824 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068018990 con 0x7ff07c102170 2024-06-03T01:21:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.510+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c1a6ea0 con 0x7ff07c102170 2024-06-03T01:21:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.514+0000 7ff085236740 1 Processor -- start 2024-06-03T01:21:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.514+0000 7ff085236740 1 -- start start 2024-06-03T01:21:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.514+0000 7ff073e00640 1 -- 172.21.15.94:0/2562727824 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff068010ac0 con 0x7ff07c102170 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff085236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f16acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff085236740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16b200 0x55be7f16b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff085236740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f178370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff085236740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be7f0cf420 con 0x55be7f175f80 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff085236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be7f076c50 con 0x55be7f16c770 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff085236740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55be7f0cb3e0 con 0x55be7f16b200 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f16acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f16acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:50546/0 (socket says 172.21.15.94:50546) 2024-06-03T01:21:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07be00640 1 -- 172.21.15.94:0/3614681205 learned_addr learned my addr 172.21.15.94:0/3614681205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07b400640 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16b200 0x55be7f16b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07be00640 1 -- 172.21.15.94:0/3614681205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16b200 msgr2=0x55be7f16b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07aa00640 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f178370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07be00640 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16b200 0x55be7f16b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.515+0000 7ff07be00640 1 -- 172.21.15.94:0/3614681205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 msgr2=0x55be7f178370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff07be00640 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f178370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff07be00640 1 -- 172.21.15.94:0/3614681205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7ede3210 con 0x55be7f16c770 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff07b400640 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16b200 0x55be7f16b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff07aa00640 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f178370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:21:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff07be00640 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f16acc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0740107d0 tx=0x7ff074010c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff0b1176634a6e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff070c00640 1 -- 172.21.15.94:0/3614681205 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074003b70 con 0x55be7f16c770 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff070c00640 1 -- 172.21.15.94:0/3614681205 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff074010e80 con 0x55be7f16c770 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.516+0000 7ff070c00640 1 -- 172.21.15.94:0/3614681205 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074004770 con 0x55be7f16c770 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.517+0000 7ff073e00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff0600a2d70 0x7ff0600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.517+0000 7ff073e00640 1 -- 172.21.15.94:0/2562727824 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7ff0680cfae0 con 0x7ff07c102170 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.517+0000 7ff07b400640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff0600a2d70 0x7ff0600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.517+0000 7ff085236740 1 -- 172.21.15.94:0/3614681205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 msgr2=0x55be7f16acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.517+0000 7ff085236740 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f16acc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0740107d0 tx=0x7ff074010c90 comp rx=0 tx=0).stop 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 -- 172.21.15.94:0/3614681205 shutdown_connections 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f178370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16b200 0x55be7f16b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 --2- 172.21.15.94:0/3614681205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f16acc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 -- 172.21.15.94:0/3614681205 >> 172.21.15.94:0/3614681205 conn(0x55be7f133f80 msgr2=0x55be7f1677f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 -- 172.21.15.94:0/3614681205 shutdown_connections 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 -- 172.21.15.94:0/3614681205 wait complete. 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff07b400640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff0600a2d70 0x7ff0600a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009900 tx=0x7ff06c0058d0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.518+0000 7ff085236740 1 -- start start 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 1 Processor -- start 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16b200 0x55be7f172fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16c770 0x55be7f173520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f173a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55be7f178d50 con 0x55be7f175f80 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55be7f08a0b0 con 0x55be7f16b200 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55be7f178ed0 con 0x55be7f16c770 2024-06-03T01:21:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.519+0000 7ff085236740 10 client.0 _ll_register_callbacks cb 0x55be7f075f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16b200 0x55be7f172fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16c770 0x55be7f173520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16b200 0x55be7f172fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:50562/0 (socket says 172.21.15.94:50562) 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 -- 172.21.15.94:0/2678167010 learned_addr learned my addr 172.21.15.94:0/2678167010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 -- 172.21.15.94:0/2678167010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16c770 msgr2=0x55be7f173520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16c770 0x55be7f173520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 -- 172.21.15.94:0/2678167010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 msgr2=0x55be7f173a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f173a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 -- 172.21.15.94:0/2678167010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7eeaf620 con 0x55be7f16b200 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07aa00640 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16c770 0x55be7f173520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff07b400640 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16b200 0x55be7f172fe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009930 tx=0x7ff06c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270663d88cf15169 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.520+0000 7ff065600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:21:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.521+0000 7ff065600640 1 -- 172.21.15.94:0/2678167010 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c04c070 con 0x55be7f16b200 2024-06-03T01:21:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.521+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be7f0dcb50 con 0x55be7f16b200 2024-06-03T01:21:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.521+0000 7ff065600640 1 -- 172.21.15.94:0/2678167010 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7ff06c003ac0 con 0x55be7f16b200 2024-06-03T01:21:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.521+0000 7ff065600640 1 -- 172.21.15.94:0/2678167010 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7ff06c042070 con 0x55be7f16b200 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.521+0000 7ff085236740 10 client.26216 fetch_fsmap learned FSMap version 56 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.521+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55be7f1850d0 con 0x55be7f16b200 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.521+0000 7ff065600640 1 -- 172.21.15.94:0/2678167010 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55be7f0dcb50 con 0x55be7f16b200 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff065600640 1 -- 172.21.15.94:0/2678167010 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7ff06c03c440 con 0x55be7f16b200 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff085236740 10 client.26216 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff085236740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff085236740 20 client.26216 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff085236740 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1865e0 0x55be7f188a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff085236740 4 client.26216 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be7f0d6ea0 con 0x55be7f1865e0 2024-06-03T01:21:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff07aa00640 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1865e0 0x55be7f188a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff07aa00640 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1865e0 0x55be7f188a50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9521001efed680b server_cookie=195aca7cc9dcdacb in_seq=0 out_seq=0 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.522+0000 7ff065600640 10 client.26216 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.523+0000 7ff065600640 1 -- 172.21.15.94:0/2678167010 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be7f0d6ea0 con 0x55be7f1865e0 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.523+0000 7ff065600640 10 client.26216 handle_command_reply: tid=0 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.523+0000 7ff085236740 1 client.26216 shutdown 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16b200 msgr2=0x55be7f172fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16b200 0x55be7f172fe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009930 tx=0x7ff06c009260 comp rx=0 tx=0).stop 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 shutdown_connections 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f175f80 0x55be7f173a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f16c770 0x55be7f173520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1865e0 0x55be7f188a50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 --2- 172.21.15.94:0/2678167010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16b200 0x55be7f172fe0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 >> 172.21.15.94:0/2678167010 conn(0x55be7f133f80 msgr2=0x55be7f16d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff065600640 0 client.26216 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 shutdown_connections 2024-06-03T01:21:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 1 -- 172.21.15.94:0/2678167010 wait complete. 2024-06-03T01:21:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.524+0000 7ff085236740 20 client.26216 trim_cache size 0 max 16384 2024-06-03T01:21:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 Processor -- start 2024-06-03T01:21:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 -- start start 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f18e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f18e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1b0f30 con 0x55be7f16b200 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1b2bc0 con 0x55be7f16c770 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.543+0000 7ff085236740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1b1630 con 0x55be7f175f80 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f18e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f18e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:50578/0 (socket says 172.21.15.94:50578) 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 -- 172.21.15.94:0/1850014492 learned_addr learned my addr 172.21.15.94:0/1850014492 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07b400640 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f18e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07be00640 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 -- 172.21.15.94:0/1850014492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 msgr2=0x55be7f1660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 -- 172.21.15.94:0/1850014492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 msgr2=0x55be7f18e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f18e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 -- 172.21.15.94:0/1850014492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be7f171530 con 0x55be7f16c770 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07be00640 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:21:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.544+0000 7ff07aa00640 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f18e8a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff06809b760 tx=0x7ff068000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc164cb807b7c76e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.545+0000 7ff067e00640 1 -- 172.21.15.94:0/1850014492 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068099d30 con 0x55be7f16c770 2024-06-03T01:21:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.545+0000 7ff067e00640 1 -- 172.21.15.94:0/1850014492 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff068099f90 con 0x55be7f16c770 2024-06-03T01:21:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.545+0000 7ff067e00640 1 -- 172.21.15.94:0/1850014492 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06809b340 con 0x55be7f16c770 2024-06-03T01:21:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.545+0000 7ff085236740 1 -- 172.21.15.94:0/1850014492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 msgr2=0x55be7f18e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.545+0000 7ff085236740 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f18e8a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff06809b760 tx=0x7ff068000bc0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 -- 172.21.15.94:0/1850014492 shutdown_connections 2024-06-03T01:21:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f18e8a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 --2- 172.21.15.94:0/1850014492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f18e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 -- 172.21.15.94:0/1850014492 >> 172.21.15.94:0/1850014492 conn(0x55be7f133f80 msgr2=0x55be7f16e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 -- 172.21.15.94:0/1850014492 shutdown_connections 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 -- 172.21.15.94:0/1850014492 wait complete. 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 -- start start 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.546+0000 7ff085236740 1 Processor -- start 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff085236740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f1e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff085236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f1e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff085236740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff085236740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1b2460 con 0x55be7f16b200 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff085236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1b1d90 con 0x55be7f16c770 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff085236740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1aef30 con 0x55be7f175f80 2024-06-03T01:21:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff085236740 10 client.0 _ll_register_callbacks cb 0x7ff06c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff07aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f1e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff07aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f1e9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:50586/0 (socket says 172.21.15.94:50586) 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff07aa00640 1 -- 172.21.15.94:0/3826665707 learned_addr learned my addr 172.21.15.94:0/3826665707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff07b400640 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f1e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.547+0000 7ff07be00640 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff07aa00640 1 -- 172.21.15.94:0/3826665707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 msgr2=0x55be7f1797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff07aa00640 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1797a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff07aa00640 1 -- 172.21.15.94:0/3826665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 msgr2=0x55be7f1e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff07aa00640 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f1e9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff07aa00640 1 -- 172.21.15.94:0/3826665707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7f179e90 con 0x55be7f16c770 2024-06-03T01:21:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff07aa00640 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f1e9c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c6a0 tx=0x7ff068098060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57638fa8354c2072 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff065600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff065600640 1 -- 172.21.15.94:0/3826665707 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0680cd090 con 0x55be7f16c770 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff065600640 1 -- 172.21.15.94:0/3826665707 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9475+0+0 (secure 0 0 0) 0x7ff0680cd230 con 0x55be7f16c770 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff068099f90 con 0x55be7f16c770 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff065600640 1 -- 172.21.15.94:0/3826665707 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7ff06809b0e0 con 0x55be7f16c770 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff085236740 10 client.26222 fetch_fsmap learned FSMap version 56 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.548+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55be7f1f0e70 con 0x55be7f16c770 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff065600640 1 -- 172.21.15.94:0/3826665707 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff068099f90 con 0x55be7f16c770 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff065600640 1 -- 172.21.15.94:0/3826665707 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7ff06809ab50 con 0x55be7f16c770 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff085236740 10 client.26222 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:21:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff085236740 10 client.26222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:21:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff085236740 20 client.26222 populate_metadata read hostname 'smithi094' 2024-06-03T01:21:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff085236740 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1f1580 0x55be7f1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:21:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff085236740 4 client.26222 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:21:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be7f0547a0 con 0x55be7f1f1580 2024-06-03T01:21:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.549+0000 7ff07b400640 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1f1580 0x55be7f1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:21:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.550+0000 7ff07b400640 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1f1580 0x55be7f1f39a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f8000641b6c159 server_cookie=60e0f0471bfb7524 in_seq=0 out_seq=0 2024-06-03T01:21:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.550+0000 7ff065600640 10 client.26222 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff065600640 1 -- 172.21.15.94:0/3826665707 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55be7f0547a0 con 0x55be7f1f1580 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff065600640 10 client.26222 handle_command_reply: tid=0 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 client.26222 shutdown 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 msgr2=0x55be7f1e9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f1e9c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c6a0 tx=0x7ff068098060 comp rx=0 tx=0).stop 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 shutdown_connections 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55be7f175f80 0x55be7f1797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55be7f1f1580 0x55be7f1f39a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55be7f16c770 0x55be7f1e9c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 --2- 172.21.15.94:0/3826665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55be7f16b200 0x55be7f1e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff065600640 0 client.26222 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:21:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.577+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 >> 172.21.15.94:0/3826665707 conn(0x55be7f133f80 msgr2=0x55be7f1660e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 shutdown_connections 2024-06-03T01:21:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff085236740 1 -- 172.21.15.94:0/3826665707 wait complete. 2024-06-03T01:21:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff085236740 20 client.26222 trim_cache size 0 max 16384 2024-06-03T01:21:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:21:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:21:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:21:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff0600a2d70 msgr2=0x7ff0600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff0600a2d70 0x7ff0600a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009900 tx=0x7ff06c0058d0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 msgr2=0x7ff07c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:21:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 0x7ff07c19a870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9f0 tx=0x7ff06800eeb0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 shutdown_connections 2024-06-03T01:21:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7ff0600a2d70 0x7ff0600a5230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff07c1077c0 0x7ff07c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c102170 0x7ff07c19a870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.578+0000 7ff080e00640 1 --2- 172.21.15.94:0/2562727824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff07c1017b0 0x7ff07c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:21:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.579+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 >> 172.21.15.94:0/2562727824 conn(0x7ff07c0fd320 msgr2=0x7ff07c10cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:21:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.579+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 shutdown_connections 2024-06-03T01:21:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:21:51.579+0000 7ff080e00640 1 -- 172.21.15.94:0/2562727824 wait complete. 2024-06-03T01:21:52.135 INFO:tasks.workunit.client.0.smithi094.stdout: 1 253572 2.01 MB/sec execute 479 sec latency 120.017 ms 2024-06-03T01:21:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:52 smithi107 ceph-mon[36622]: pgmap v358: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 836 KiB/s wr, 66 op/s 2024-06-03T01:21:52.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:52 smithi094 ceph-mon[32119]: pgmap v358: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 836 KiB/s wr, 66 op/s 2024-06-03T01:21:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:52 smithi179 ceph-mon[37041]: pgmap v358: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 836 KiB/s wr, 66 op/s 2024-06-03T01:21:53.136 INFO:tasks.workunit.client.0.smithi094.stdout: 1 254756 2.02 MB/sec execute 480 sec latency 363.230 ms 2024-06-03T01:21:54.135 INFO:tasks.workunit.client.0.smithi094.stdout: 1 255822 2.03 MB/sec execute 481 sec latency 14.149 ms 2024-06-03T01:21:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:54 smithi107 ceph-mon[36622]: pgmap v359: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-06-03T01:21:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:54 smithi094 ceph-mon[32119]: pgmap v359: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-06-03T01:21:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:54 smithi179 ceph-mon[37041]: pgmap v359: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-06-03T01:21:55.135 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256087 2.03 MB/sec execute 482 sec latency 622.322 ms 2024-06-03T01:21:56.135 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256143 2.02 MB/sec execute 483 sec latency 1543.298 ms 2024-06-03T01:21:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:56 smithi107 ceph-mon[36622]: pgmap v360: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 861 KiB/s wr, 79 op/s 2024-06-03T01:21:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:56 smithi094 ceph-mon[32119]: pgmap v360: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 861 KiB/s wr, 79 op/s 2024-06-03T01:21:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:56 smithi179 ceph-mon[37041]: pgmap v360: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 861 KiB/s wr, 79 op/s 2024-06-03T01:21:57.135 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256143 2.02 MB/sec execute 484 sec latency 1014.525 ms 2024-06-03T01:21:58.136 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256143 2.01 MB/sec execute 485 sec latency 2014.674 ms 2024-06-03T01:21:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:21:58 smithi107 ceph-mon[36622]: pgmap v361: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-06-03T01:21:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:21:58 smithi094 ceph-mon[32119]: pgmap v361: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-06-03T01:21:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:21:58 smithi179 ceph-mon[37041]: pgmap v361: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-06-03T01:21:59.136 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256143 2.01 MB/sec execute 486 sec latency 3014.887 ms 2024-06-03T01:22:00.136 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256143 2.01 MB/sec execute 487 sec latency 4014.983 ms 2024-06-03T01:22:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:00 smithi107 ceph-mon[36622]: pgmap v362: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 848 KiB/s wr, 77 op/s 2024-06-03T01:22:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:00 smithi094 ceph-mon[32119]: pgmap v362: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 848 KiB/s wr, 77 op/s 2024-06-03T01:22:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:00 smithi179 ceph-mon[37041]: pgmap v362: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 848 KiB/s wr, 77 op/s 2024-06-03T01:22:01.136 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256346 2.00 MB/sec execute 488 sec latency 4580.061 ms 2024-06-03T01:22:02.136 INFO:tasks.workunit.client.0.smithi094.stdout: 1 256452 2.00 MB/sec execute 489 sec latency 103.828 ms 2024-06-03T01:22:02.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:02 smithi107 ceph-mon[36622]: pgmap v363: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 849 KiB/s wr, 77 op/s 2024-06-03T01:22:02.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:02 smithi094 ceph-mon[32119]: pgmap v363: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 849 KiB/s wr, 77 op/s 2024-06-03T01:22:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:02 smithi179 ceph-mon[37041]: pgmap v363: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 849 KiB/s wr, 77 op/s 2024-06-03T01:22:03.136 INFO:tasks.workunit.client.0.smithi094.stdout: 1 257113 2.00 MB/sec execute 490 sec latency 94.995 ms 2024-06-03T01:22:04.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 257113 2.00 MB/sec execute 491 sec latency 1084.845 ms 2024-06-03T01:22:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:04 smithi094 ceph-mon[32119]: pgmap v364: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-03T01:22:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:04 smithi179 ceph-mon[37041]: pgmap v364: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 -- 172.21.15.94:0/2285565068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0108200 msgr2=0x7fbcc01085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2285565068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0108200 0x7fbcc01085e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009860 tx=0x7fbcb002a340 comp rx=0 tx=0).stop 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 -- 172.21.15.94:0/2285565068 shutdown_connections 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2285565068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc010d2d0 0x7fbcc0114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2285565068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcc0108bb0 0x7fbcc010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2285565068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc0108200 0x7fbcc01085e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 -- 172.21.15.94:0/2285565068 >> 172.21.15.94:0/2285565068 conn(0x7fbcc0075f30 msgr2=0x7fbcc0078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.719+0000 7fbcc7600640 1 -- 172.21.15.94:0/2285565068 shutdown_connections 2024-06-03T01:22:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.720+0000 7fbcc7600640 1 -- 172.21.15.94:0/2285565068 wait complete. 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 Processor -- start 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 -- start start 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc0108200 0x7fbcc01a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc010d2d0 0x7fbcc01a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcc019e620 0x7fbcc019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0118360 con 0x7fbcc0108200 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01181e0 con 0x7fbcc010d2d0 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc7600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01184e0 con 0x7fbcc019e620 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.721+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc0108200 0x7fbcc01a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc0108200 0x7fbcc01a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60472/0 (socket says 172.21.15.94:60472) 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 -- 172.21.15.94:0/2774966531 learned_addr learned my addr 172.21.15.94:0/2774966531 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc010d2d0 0x7fbcc01a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc5600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcc019e620 0x7fbcc019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 -- 172.21.15.94:0/2774966531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcc019e620 msgr2=0x7fbcc019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcc019e620 0x7fbcc019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 -- 172.21.15.94:0/2774966531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc010d2d0 msgr2=0x7fbcc01a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc010d2d0 0x7fbcc01a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 -- 172.21.15.94:0/2774966531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc019f140 con 0x7fbcc0108200 2024-06-03T01:22:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.722+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc0108200 0x7fbcc01a4420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0035040 tx=0x7fbcb0033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.723+0000 7fbcbd600640 1 -- 172.21.15.94:0/2774966531 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0037040 con 0x7fbcc0108200 2024-06-03T01:22:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.723+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc019f3d0 con 0x7fbcc0108200 2024-06-03T01:22:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.723+0000 7fbcbd600640 1 -- 172.21.15.94:0/2774966531 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb0040070 con 0x7fbcc0108200 2024-06-03T01:22:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.723+0000 7fbcbd600640 1 -- 172.21.15.94:0/2774966531 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00039b0 con 0x7fbcc0108200 2024-06-03T01:22:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.723+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01ab5d0 con 0x7fbcc0108200 2024-06-03T01:22:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 Processor -- start 2024-06-03T01:22:04.729 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 -- start start 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcbd600640 1 -- 172.21.15.94:0/2774966531 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbcb00305e0 con 0x7fbcc0108200 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a91188b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91189040 0x562a911894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91193dc0 0x562a911961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a910ed420 con 0x562a91193dc0 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a91094c50 con 0x562a91189040 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcca438740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a910e93e0 con 0x562a9118a5b0 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91193dc0 0x562a911961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91193dc0 0x562a911961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60476/0 (socket says 172.21.15.94:60476) 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcbfe00640 1 -- 172.21.15.94:0/226155062 learned_addr learned my addr 172.21.15.94:0/226155062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91189040 0x562a911894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcc5600640 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a91188b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcc4c00640 1 -- 172.21.15.94:0/226155062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 msgr2=0x562a91188b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a91188b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcc4c00640 1 -- 172.21.15.94:0/226155062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91193dc0 msgr2=0x562a911961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91193dc0 0x562a911961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.728+0000 7fbcc4c00640 1 -- 172.21.15.94:0/226155062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a90e01210 con 0x562a91189040 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcc5600640 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a91188b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91189040 0x562a911894c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00eca90 tx=0x7fbcb00f5b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87982e6d11c18923 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcbd600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbca40a2e20 0x7fbca40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcbd600640 1 -- 172.21.15.94:0/2774966531 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9563+0+0 (secure 0 0 0) 0x7fbcb00ef670 con 0x7fbcc0108200 2024-06-03T01:22:04.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcc5600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbca40a2e20 0x7fbca40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcb5c00640 1 -- 172.21.15.94:0/226155062 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00fe070 con 0x562a91189040 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcb5c00640 1 -- 172.21.15.94:0/226155062 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb00bbf90 con 0x562a91189040 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.729+0000 7fbcb5c00640 1 -- 172.21.15.94:0/226155062 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00bb660 con 0x562a91189040 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 -- 172.21.15.94:0/226155062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91189040 msgr2=0x562a911894c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91189040 0x562a911894c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00eca90 tx=0x7fbcb00f5b40 comp rx=0 tx=0).stop 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcc5600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbca40a2e20 0x7fbca40a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009fd0 tx=0x7fbcb8009500 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 -- 172.21.15.94:0/226155062 shutdown_connections 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91193dc0 0x562a911961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91189040 0x562a911894c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 --2- 172.21.15.94:0/226155062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a91188b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 -- 172.21.15.94:0/226155062 >> 172.21.15.94:0/226155062 conn(0x562a91151f80 msgr2=0x562a91185630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 -- 172.21.15.94:0/226155062 shutdown_connections 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.730+0000 7fbcca438740 1 -- 172.21.15.94:0/226155062 wait complete. 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 -- start start 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 Processor -- start 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a91189040 0x562a91190e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a9118a5b0 0x562a91191350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a91191890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a911968b0 con 0x562a9118a5b0 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a910a80b0 con 0x562a91193dc0 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a91196a30 con 0x562a91189040 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.731+0000 7fbcca438740 10 client.0 _ll_register_callbacks cb 0x562a91093f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a91189040 0x562a91190e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a91189040 0x562a91190e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:42728/0 (socket says 172.21.15.94:42728) 2024-06-03T01:22:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 -- 172.21.15.94:0/4246299036 learned_addr learned my addr 172.21.15.94:0/4246299036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a9118a5b0 0x562a91191350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc5600640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a91191890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 -- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 msgr2=0x562a91191890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a91191890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 -- 172.21.15.94:0/4246299036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a9118a5b0 msgr2=0x562a91191350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a9118a5b0 0x562a91191350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 -- 172.21.15.94:0/4246299036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a90ecd620 con 0x562a91189040 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc5600640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a91191890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a91189040 0x562a91190e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00ecfb0 tx=0x7fbcb0030550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c3a1eba46baeff server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcaa000640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.732+0000 7fbcaa000640 1 -- 172.21.15.94:0/4246299036 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00fe070 con 0x562a91189040 2024-06-03T01:22:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcaa000640 1 -- 172.21.15.94:0/4246299036 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9563+0+0 (secure 0 0 0) 0x7fbcb00a60f0 con 0x562a91189040 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb00bbf90 con 0x562a91189040 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcaa000640 1 -- 172.21.15.94:0/4246299036 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fbcb00bb1f0 con 0x562a91189040 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 10 client.26242 fetch_fsmap learned FSMap version 56 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x562a911968b0 con 0x562a91189040 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcaa000640 1 -- 172.21.15.94:0/4246299036 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb00bbf90 con 0x562a91189040 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcaa000640 1 -- 172.21.15.94:0/4246299036 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7fbcb003a070 con 0x562a91189040 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 10 client.26242 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 20 client.26242 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a911a3f20 0x562a911a6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 4 client.26242 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:22:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.733+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a910fab50 con 0x562a911a3f20 2024-06-03T01:22:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.734+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a911a3f20 0x562a911a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.734+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a911a3f20 0x562a911a6380 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12578ecb1b617c25 server_cookie=dddb48e857b580c4 in_seq=0 out_seq=0 2024-06-03T01:22:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.734+0000 7fbcaa000640 10 client.26242 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcaa000640 1 -- 172.21.15.94:0/4246299036 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a910fab50 con 0x562a911a3f20 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcaa000640 10 client.26242 handle_command_reply: tid=0 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 client.26242 shutdown 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a91189040 msgr2=0x562a91190e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a91189040 0x562a91190e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00ecfb0 tx=0x7fbcb0030550 comp rx=0 tx=0).stop 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 shutdown_connections 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a911a3f20 0x562a911a6380 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a91191890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a9118a5b0 0x562a91191350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcaa000640 0 client.26242 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 --2- 172.21.15.94:0/4246299036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a91189040 0x562a91190e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.735+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 >> 172.21.15.94:0/4246299036 conn(0x562a91151f80 msgr2=0x562a9118b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.736+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 shutdown_connections 2024-06-03T01:22:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.736+0000 7fbcca438740 1 -- 172.21.15.94:0/4246299036 wait complete. 2024-06-03T01:22:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.736+0000 7fbcca438740 20 client.26242 trim_cache size 0 max 16384 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.752+0000 7fbcca438740 1 Processor -- start 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.752+0000 7fbcca438740 1 -- start start 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.752+0000 7fbcca438740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a911905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.752+0000 7fbcca438740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9118b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.752+0000 7fbcca438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.752+0000 7fbcca438740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01b54e0 con 0x562a91189040 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.752+0000 7fbcca438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01b7170 con 0x562a91193dc0 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcca438740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01b5be0 con 0x562a9118a5b0 2024-06-03T01:22:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a911905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a911905c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60502/0 (socket says 172.21.15.94:60502) 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 -- 172.21.15.94:0/1664810259 learned_addr learned my addr 172.21.15.94:0/1664810259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9118b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc5600640 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 -- 172.21.15.94:0/1664810259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 msgr2=0x562a9118b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9118b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 -- 172.21.15.94:0/1664810259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 msgr2=0x562a9118b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 -- 172.21.15.94:0/1664810259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a91189b10 con 0x562a91189040 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc5600640 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:22:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.753+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a911905c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00b9b40 tx=0x7fbcb00b9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7655dca024b967e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcb5200640 1 -- 172.21.15.94:0/1664810259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00fe050 con 0x562a91189040 2024-06-03T01:22:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcb5200640 1 -- 172.21.15.94:0/1664810259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb00bbf90 con 0x562a91189040 2024-06-03T01:22:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcb5200640 1 -- 172.21.15.94:0/1664810259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00bb3d0 con 0x562a91189040 2024-06-03T01:22:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcca438740 1 -- 172.21.15.94:0/1664810259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 msgr2=0x562a911905c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcca438740 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a911905c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00b9b40 tx=0x7fbcb00b9da0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcca438740 1 -- 172.21.15.94:0/1664810259 shutdown_connections 2024-06-03T01:22:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcca438740 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcca438740 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9118b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcca438740 1 --2- 172.21.15.94:0/1664810259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a911905c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.754+0000 7fbcca438740 1 -- 172.21.15.94:0/1664810259 >> 172.21.15.94:0/1664810259 conn(0x562a91151f80 msgr2=0x562a9118aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.755+0000 7fbcca438740 1 -- 172.21.15.94:0/1664810259 shutdown_connections 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.755+0000 7fbcca438740 1 -- 172.21.15.94:0/1664810259 wait complete. 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.755+0000 7fbcca438740 1 -- start start 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.755+0000 7fbcca438740 1 Processor -- start 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcca438740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a9119bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcca438740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9119c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcca438740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcca438740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01b6a10 con 0x562a91189040 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcca438740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01b6340 con 0x562a91193dc0 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcca438740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01b34e0 con 0x562a9118a5b0 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcca438740 10 client.0 _ll_register_callbacks cb 0x7fbcac009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a9119bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a9119bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60504/0 (socket says 172.21.15.94:60504) 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 -- 172.21.15.94:0/740546461 learned_addr learned my addr 172.21.15.94:0/740546461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc5600640 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9119c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 -- 172.21.15.94:0/740546461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 msgr2=0x562a9119c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9119c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 -- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 msgr2=0x562a9118d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 -- 172.21.15.94:0/740546461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9119c8f0 con 0x562a91189040 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcc4c00640 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a9119bcc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00bfc40 tx=0x7fbcb00f5b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caede7c466871c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.756+0000 7fbcaa000640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcaa000640 1 -- 172.21.15.94:0/740546461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00fe070 con 0x562a91189040 2024-06-03T01:22:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcaa000640 1 -- 172.21.15.94:0/740546461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9563+0+0 (secure 0 0 0) 0x7fbcb00bbcf0 con 0x562a91189040 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb00bbf90 con 0x562a91189040 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcaa000640 1 -- 172.21.15.94:0/740546461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fbcb00bb3d0 con 0x562a91189040 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 10 client.16122 fetch_fsmap learned FSMap version 56 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x562a9119c8f0 con 0x562a91189040 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcaa000640 1 -- 172.21.15.94:0/740546461 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb00bbf90 con 0x562a91189040 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcaa000640 1 -- 172.21.15.94:0/740546461 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7fbcb003a070 con 0x562a91189040 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 10 client.16122 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 20 client.16122 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a9120e420 0x562a91210840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 4 client.16122 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.757+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a910f4ea0 con 0x562a9120e420 2024-06-03T01:22:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.758+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a9120e420 0x562a91210840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.758+0000 7fbcbfe00640 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a9120e420 0x562a91210840 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aebc3486d14e9b9 server_cookie=ec24f9d5ae8c3c93 in_seq=0 out_seq=0 2024-06-03T01:22:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.758+0000 7fbcaa000640 10 client.16122 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcaa000640 1 -- 172.21.15.94:0/740546461 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a910f4ea0 con 0x562a9120e420 2024-06-03T01:22:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcaa000640 10 client.16122 handle_command_reply: tid=0 2024-06-03T01:22:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 client.16122 shutdown 2024-06-03T01:22:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 msgr2=0x562a9119bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a9119bcc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00bfc40 tx=0x7fbcb00f5b40 comp rx=0 tx=0).stop 2024-06-03T01:22:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 shutdown_connections 2024-06-03T01:22:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a9120e420 0x562a91210840 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a91193dc0 0x562a9118d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9118a5b0 0x562a9119c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 --2- 172.21.15.94:0/740546461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a91189040 0x562a9119bcc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 >> 172.21.15.94:0/740546461 conn(0x562a91151f80 msgr2=0x562a9118b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcaa000640 0 client.16122 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.781+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 shutdown_connections 2024-06-03T01:22:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.782+0000 7fbcca438740 1 -- 172.21.15.94:0/740546461 wait complete. 2024-06-03T01:22:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.782+0000 7fbcca438740 20 client.16122 trim_cache size 0 max 16384 2024-06-03T01:22:04.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:04 smithi107 ceph-mon[36622]: pgmap v364: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-03T01:22:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:22:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:22:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbca40a2e20 msgr2=0x7fbca40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbca40a2e20 0x7fbca40a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009fd0 tx=0x7fbcb8009500 comp rx=0 tx=0).stop 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc0108200 msgr2=0x7fbcc01a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc0108200 0x7fbcc01a4420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0035040 tx=0x7fbcb0033b60 comp rx=0 tx=0).stop 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 shutdown_connections 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fbca40a2e20 0x7fbca40a52e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcc019e620 0x7fbcc019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcc010d2d0 0x7fbcc01a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 --2- 172.21.15.94:0/2774966531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbcc0108200 0x7fbcc01a4420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 >> 172.21.15.94:0/2774966531 conn(0x7fbcc0075f30 msgr2=0x7fbcc0078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 shutdown_connections 2024-06-03T01:22:04.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:04.783+0000 7fbcc7600640 1 -- 172.21.15.94:0/2774966531 wait complete. 2024-06-03T01:22:05.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 257113 2.00 MB/sec execute 492 sec latency 2084.982 ms 2024-06-03T01:22:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:05 smithi094 ceph-mon[32119]: osdmap e112: 12 total, 12 up, 12 in 2024-06-03T01:22:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:05 smithi179 ceph-mon[37041]: osdmap e112: 12 total, 12 up, 12 in 2024-06-03T01:22:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:05 smithi107 ceph-mon[36622]: osdmap e112: 12 total, 12 up, 12 in 2024-06-03T01:22:06.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 257113 1.99 MB/sec execute 493 sec latency 3085.167 ms 2024-06-03T01:22:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:06 smithi094 ceph-mon[32119]: pgmap v366: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 777 KiB/s wr, 111 op/s 2024-06-03T01:22:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:06 smithi179 ceph-mon[37041]: pgmap v366: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 777 KiB/s wr, 111 op/s 2024-06-03T01:22:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:06 smithi107 ceph-mon[36622]: pgmap v366: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 777 KiB/s wr, 111 op/s 2024-06-03T01:22:07.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 257113 1.99 MB/sec execute 494 sec latency 4085.300 ms 2024-06-03T01:22:08.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 258181 2.00 MB/sec execute 495 sec latency 4329.642 ms 2024-06-03T01:22:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:08 smithi094 ceph-mon[32119]: pgmap v367: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 360 KiB/s wr, 93 op/s 2024-06-03T01:22:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:08 smithi179 ceph-mon[37041]: pgmap v367: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 360 KiB/s wr, 93 op/s 2024-06-03T01:22:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:08 smithi107 ceph-mon[36622]: pgmap v367: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 360 KiB/s wr, 93 op/s 2024-06-03T01:22:09.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 259294 2.01 MB/sec execute 496 sec latency 13.918 ms 2024-06-03T01:22:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:09 smithi107 ceph-mon[36622]: osdmap e113: 12 total, 12 up, 12 in 2024-06-03T01:22:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:09 smithi107 ceph-mon[36622]: pgmap v369: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 449 KiB/s wr, 117 op/s 2024-06-03T01:22:10.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:09 smithi094 ceph-mon[32119]: osdmap e113: 12 total, 12 up, 12 in 2024-06-03T01:22:10.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:09 smithi094 ceph-mon[32119]: pgmap v369: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 449 KiB/s wr, 117 op/s 2024-06-03T01:22:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:09 smithi179 ceph-mon[37041]: osdmap e113: 12 total, 12 up, 12 in 2024-06-03T01:22:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:09 smithi179 ceph-mon[37041]: pgmap v369: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 449 KiB/s wr, 117 op/s 2024-06-03T01:22:10.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 260064 2.01 MB/sec execute 497 sec latency 20.043 ms 2024-06-03T01:22:11.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 260661 2.01 MB/sec execute 498 sec latency 482.227 ms 2024-06-03T01:22:12.137 INFO:tasks.workunit.client.0.smithi094.stdout: 1 260661 2.01 MB/sec execute 499 sec latency 1482.365 ms 2024-06-03T01:22:12.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:12 smithi107 ceph-mon[36622]: pgmap v370: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 177 KiB/s wr, 46 op/s 2024-06-03T01:22:12.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:12 smithi094 ceph-mon[32119]: pgmap v370: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 177 KiB/s wr, 46 op/s 2024-06-03T01:22:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:12 smithi179 ceph-mon[37041]: pgmap v370: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 177 KiB/s wr, 46 op/s 2024-06-03T01:22:13.138 INFO:tasks.workunit.client.0.smithi094.stdout: 1 261753 2.01 MB/sec execute 500 sec latency 1725.800 ms 2024-06-03T01:22:14.138 INFO:tasks.workunit.client.0.smithi094.stdout: 1 262839 2.03 MB/sec execute 501 sec latency 18.192 ms 2024-06-03T01:22:14.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:14 smithi107 ceph-mon[36622]: pgmap v371: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 615 KiB/s wr, 112 op/s 2024-06-03T01:22:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:14 smithi094 ceph-mon[32119]: pgmap v371: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 615 KiB/s wr, 112 op/s 2024-06-03T01:22:14.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:14 smithi179 ceph-mon[37041]: pgmap v371: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 615 KiB/s wr, 112 op/s 2024-06-03T01:22:15.138 INFO:tasks.workunit.client.0.smithi094.stdout: 1 263303 2.02 MB/sec execute 502 sec latency 240.204 ms 2024-06-03T01:22:16.138 INFO:tasks.workunit.client.0.smithi094.stdout: 1 263378 2.02 MB/sec execute 503 sec latency 1161.268 ms 2024-06-03T01:22:16.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:16 smithi107 ceph-mon[36622]: pgmap v372: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 540 KiB/s wr, 98 op/s 2024-06-03T01:22:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:16 smithi094 ceph-mon[32119]: pgmap v372: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 540 KiB/s wr, 98 op/s 2024-06-03T01:22:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:16 smithi179 ceph-mon[37041]: pgmap v372: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 540 KiB/s wr, 98 op/s 2024-06-03T01:22:17.138 INFO:tasks.workunit.client.0.smithi094.stdout: 1 264204 2.02 MB/sec execute 504 sec latency 153.273 ms 2024-06-03T01:22:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 -- 172.21.15.94:0/3663681316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 msgr2=0x7f953c1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 --2- 172.21.15.94:0/3663681316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 0x7f953c1041d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f952c009900 tx=0x7f952c02a3f0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 -- 172.21.15.94:0/3663681316 shutdown_connections 2024-06-03T01:22:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 --2- 172.21.15.94:0/3663681316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 0x7f953c110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 --2- 172.21.15.94:0/3663681316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c1047a0 0x7f953c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 --2- 172.21.15.94:0/3663681316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 0x7f953c1041d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 -- 172.21.15.94:0/3663681316 >> 172.21.15.94:0/3663681316 conn(0x7f953c0ff690 msgr2=0x7f953c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 -- 172.21.15.94:0/3663681316 shutdown_connections 2024-06-03T01:22:17.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.936+0000 7f9541400640 1 -- 172.21.15.94:0/3663681316 wait complete. 2024-06-03T01:22:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.937+0000 7f9541400640 1 Processor -- start 2024-06-03T01:22:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.937+0000 7f9541400640 1 -- start start 2024-06-03T01:22:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.937+0000 7f9541400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 0x7f953c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9541400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 0x7f953c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9541400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c19ec60 0x7f953c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9541400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f953c113fc0 con 0x7f953c108ec0 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9541400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f953c113e40 con 0x7f953c19ec60 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9541400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f953c114140 con 0x7f953c103df0 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 0x7f953c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9539e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 0x7f953c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c19ec60 0x7f953c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c19ec60 0x7f953c19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51468/0 (socket says 172.21.15.94:51468) 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9539e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 0x7f953c19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47480/0 (socket says 172.21.15.94:47480) 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 -- 172.21.15.94:0/2424164322 learned_addr learned my addr 172.21.15.94:0/2424164322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 -- 172.21.15.94:0/2424164322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 msgr2=0x7f953c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 0x7f953c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 -- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 msgr2=0x7f953c19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 0x7f953c19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953b200640 1 -- 172.21.15.94:0/2424164322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c19f750 con 0x7f953c19ec60 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f9539e00640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 0x7f953c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.938+0000 7f953a800640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 0x7f953c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.939+0000 7f953b200640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c19ec60 0x7f953c19f110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f953400eb10 tx=0x7f953400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.939+0000 7f9533400640 1 -- 172.21.15.94:0/2424164322 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534004b20 con 0x7f953c19ec60 2024-06-03T01:22:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.939+0000 7f9533400640 1 -- 172.21.15.94:0/2424164322 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9534004cc0 con 0x7f953c19ec60 2024-06-03T01:22:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.939+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953c19fa40 con 0x7f953c19ec60 2024-06-03T01:22:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.939+0000 7f9533400640 1 -- 172.21.15.94:0/2424164322 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534010760 con 0x7f953c19ec60 2024-06-03T01:22:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.939+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953c1a71e0 con 0x7f953c19ec60 2024-06-03T01:22:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.942+0000 7f9533400640 1 -- 172.21.15.94:0/2424164322 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9534005020 con 0x7f953c19ec60 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.942+0000 7f9544397740 1 Processor -- start 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.942+0000 7f9544397740 1 -- start start 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9544397740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848adbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9544397740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848adc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9544397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848ae91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9544397740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563848a40420 con 0x563848add5b0 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9544397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5638489e7c50 con 0x563848ae6dc0 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9544397740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563848a3c3e0 con 0x563848adc040 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9539e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848adbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9539e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848adbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47484/0 (socket says 172.21.15.94:47484) 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f953a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848adc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9539e00640 1 -- 172.21.15.94:0/1267407309 learned_addr learned my addr 172.21.15.94:0/1267407309 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.943+0000 7f9539e00640 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848ae91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f953a800640 1 -- 172.21.15.94:0/1267407309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 msgr2=0x563848ae91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f953a800640 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848ae91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f953a800640 1 -- 172.21.15.94:0/1267407309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 msgr2=0x563848adbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f953a800640 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848adbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f953a800640 1 -- 172.21.15.94:0/1267407309 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563848754210 con 0x563848adc040 2024-06-03T01:22:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f9539e00640 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848adbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f953a800640 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848adc4c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f952c02a910 tx=0x7f952c005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=142c39cc765cb0f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f9533400640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95200a2da0 0x7f95200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f9533400640 1 -- 172.21.15.94:0/2424164322 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f95340cfd60 con 0x7f953c19ec60 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.944+0000 7f9527400640 1 -- 172.21.15.94:0/1267407309 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c040070 con 0x563848adc040 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.945+0000 7f9527400640 1 -- 172.21.15.94:0/1267407309 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f952c004760 con 0x563848adc040 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.945+0000 7f9527400640 1 -- 172.21.15.94:0/1267407309 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c035040 con 0x563848adc040 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.945+0000 7f9539e00640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95200a2da0 0x7f95200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.945+0000 7f9539e00640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95200a2da0 0x7f95200a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x563848ad97c0 tx=0x7f9528009b30 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.945+0000 7f9544397740 1 -- 172.21.15.94:0/1267407309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 msgr2=0x563848adc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.945+0000 7f9544397740 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848adc4c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f952c02a910 tx=0x7f952c005de0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 -- 172.21.15.94:0/1267407309 shutdown_connections 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848ae91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848adc4c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 --2- 172.21.15.94:0/1267407309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848adbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 -- 172.21.15.94:0/1267407309 >> 172.21.15.94:0/1267407309 conn(0x563848aa4f80 msgr2=0x563848ad8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 -- 172.21.15.94:0/1267407309 shutdown_connections 2024-06-03T01:22:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 -- 172.21.15.94:0/1267407309 wait complete. 2024-06-03T01:22:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.946+0000 7f9544397740 1 -- start start 2024-06-03T01:22:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 1 Processor -- start 2024-06-03T01:22:17.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848adc040 0x563848ae3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848add5b0 0x563848ae43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848ae6dc0 0x563848ae48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x563848ae9b30 con 0x563848ae6dc0 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5638489fb0b0 con 0x563848adc040 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563848ae9cb0 con 0x563848add5b0 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f9544397740 10 client.0 _ll_register_callbacks cb 0x5638489e6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.947+0000 7f953a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848adc040 0x563848ae3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848adc040 0x563848ae3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51488/0 (socket says 172.21.15.94:51488) 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 -- 172.21.15.94:0/1054007590 learned_addr learned my addr 172.21.15.94:0/1054007590 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f9539e00640 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848add5b0 0x563848ae43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953b200640 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848ae6dc0 0x563848ae48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 -- 172.21.15.94:0/1054007590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848add5b0 msgr2=0x563848ae43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848add5b0 0x563848ae43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 -- 172.21.15.94:0/1054007590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848ae6dc0 msgr2=0x563848ae48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848ae6dc0 0x563848ae48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 -- 172.21.15.94:0/1054007590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563848820620 con 0x563848adc040 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f953a800640 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848adc040 0x563848ae3e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f952c004b70 tx=0x7f952c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6fe7b531eaeee7b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.948+0000 7f951be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.949+0000 7f951be00640 1 -- 172.21.15.94:0/1054007590 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c040070 con 0x563848adc040 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.949+0000 7f951be00640 1 -- 172.21.15.94:0/1054007590 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f952c005250 con 0x563848adc040 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.949+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f952c004760 con 0x563848adc040 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.949+0000 7f951be00640 1 -- 172.21.15.94:0/1054007590 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f952c057050 con 0x563848adc040 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.949+0000 7f9544397740 10 client.26249 fetch_fsmap learned FSMap version 56 2024-06-03T01:22:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.949+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x563848af5f70 con 0x563848adc040 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.949+0000 7f951be00640 1 -- 172.21.15.94:0/1054007590 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f952c004760 con 0x563848adc040 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f951be00640 1 -- 172.21.15.94:0/1054007590 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f952c0035f0 con 0x563848adc040 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f9544397740 10 client.26249 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f9544397740 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f9544397740 20 client.26249 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f9544397740 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848af72c0 0x563848af96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f9544397740 4 client.26249 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563848a4db50 con 0x563848af72c0 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.950+0000 7f9539e00640 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848af72c0 0x563848af96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.951+0000 7f9539e00640 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848af72c0 0x563848af96b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a0650d761479d server_cookie=b13285c2930ff4ac in_seq=0 out_seq=0 2024-06-03T01:22:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.951+0000 7f951be00640 10 client.26249 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f951be00640 1 -- 172.21.15.94:0/1054007590 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563848a4db50 con 0x563848af72c0 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f951be00640 10 client.26249 handle_command_reply: tid=0 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 client.26249 shutdown 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848adc040 msgr2=0x563848ae3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848adc040 0x563848ae3e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f952c004b70 tx=0x7f952c000bc0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 shutdown_connections 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848ae6dc0 0x563848ae48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848add5b0 0x563848ae43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848af72c0 0x563848af96b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 --2- 172.21.15.94:0/1054007590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848adc040 0x563848ae3e70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f951be00640 0 client.26249 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 >> 172.21.15.94:0/1054007590 conn(0x563848aa4f80 msgr2=0x563848ade4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 shutdown_connections 2024-06-03T01:22:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 1 -- 172.21.15.94:0/1054007590 wait complete. 2024-06-03T01:22:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.952+0000 7f9544397740 20 client.26249 trim_cache size 0 max 16384 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 Processor -- start 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 -- start start 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848ad7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848afec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1b5340 con 0x563848add5b0 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1b6fd0 con 0x563848ae6dc0 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f9544397740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1b5a40 con 0x563848adc040 2024-06-03T01:22:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.971+0000 7f953a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848ad7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51490/0 (socket says 172.21.15.94:51490) 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 -- 172.21.15.94:0/2275228807 learned_addr learned my addr 172.21.15.94:0/2275228807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848ad7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53104/0 (socket says 172.21.15.94:53104) 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f9539e00640 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848afec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 -- 172.21.15.94:0/2275228807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 msgr2=0x563848ad7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848ad7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 -- 172.21.15.94:0/2275228807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 msgr2=0x563848afec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848afec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 -- 172.21.15.94:0/2275228807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563848adcb10 con 0x563848ae6dc0 2024-06-03T01:22:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f953b200640 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aff1c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f953409bb60 tx=0x7f953409b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee5e91cf4bf8a51 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f9526a00640 1 -- 172.21.15.94:0/2275228807 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953409be60 con 0x563848ae6dc0 2024-06-03T01:22:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f9526a00640 1 -- 172.21.15.94:0/2275228807 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95340a0210 con 0x563848ae6dc0 2024-06-03T01:22:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.972+0000 7f9526a00640 1 -- 172.21.15.94:0/2275228807 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953409c320 con 0x563848ae6dc0 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.973+0000 7f9544397740 1 -- 172.21.15.94:0/2275228807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 msgr2=0x563848aff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.973+0000 7f9544397740 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aff1c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f953409bb60 tx=0x7f953409b9a0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.973+0000 7f9544397740 1 -- 172.21.15.94:0/2275228807 shutdown_connections 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.973+0000 7f9544397740 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aff1c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.973+0000 7f9544397740 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848afec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.973+0000 7f9544397740 1 --2- 172.21.15.94:0/2275228807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848ad7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.973+0000 7f9544397740 1 -- 172.21.15.94:0/2275228807 >> 172.21.15.94:0/2275228807 conn(0x563848aa4f80 msgr2=0x563848ad6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.974+0000 7f9544397740 1 -- 172.21.15.94:0/2275228807 shutdown_connections 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.974+0000 7f9544397740 1 -- 172.21.15.94:0/2275228807 wait complete. 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.974+0000 7f9544397740 1 -- start start 2024-06-03T01:22:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.974+0000 7f9544397740 1 Processor -- start 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9544397740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848aeae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9544397740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848aeb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9544397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aeb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9544397740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1b6870 con 0x563848add5b0 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9544397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1b61a0 con 0x563848ae6dc0 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9544397740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1b3340 con 0x563848adc040 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9544397740 10 client.0 _ll_register_callbacks cb 0x7f9528009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f953b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aeb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f953a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848aeae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f953b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aeb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51502/0 (socket says 172.21.15.94:51502) 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f953a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848aeae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53108/0 (socket says 172.21.15.94:53108) 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f953b200640 1 -- 172.21.15.94:0/2629814603 learned_addr learned my addr 172.21.15.94:0/2629814603 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.975+0000 7f9539e00640 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848aeb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f953b200640 1 -- 172.21.15.94:0/2629814603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 msgr2=0x563848aeae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f953b200640 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848aeae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f953b200640 1 -- 172.21.15.94:0/2629814603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 msgr2=0x563848aeb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f953b200640 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848aeb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f953b200640 1 -- 172.21.15.94:0/2629814603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563848b60db0 con 0x563848ae6dc0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f953a800640 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848aeae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f953b200640 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aeb900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f95340a45a0 tx=0x7f9534005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e015554130ce9fc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f951be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f951be00640 1 -- 172.21.15.94:0/2629814603 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534081020 con 0x563848ae6dc0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f951be00640 1 -- 172.21.15.94:0/2629814603 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f953409c2c0 con 0x563848ae6dc0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95340a0210 con 0x563848ae6dc0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f951be00640 1 -- 172.21.15.94:0/2629814603 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f95340ac050 con 0x563848ae6dc0 2024-06-03T01:22:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f9544397740 10 client.26255 fetch_fsmap learned FSMap version 56 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.976+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x563848b620a0 con 0x563848ae6dc0 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f951be00640 1 -- 172.21.15.94:0/2629814603 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95340a0210 con 0x563848ae6dc0 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f951be00640 1 -- 172.21.15.94:0/2629814603 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f9534005430 con 0x563848ae6dc0 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f9544397740 10 client.26255 fetch_fsmap finished waiting for FSMap version 56 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f9544397740 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f9544397740 20 client.26255 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f9544397740 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848b62850 0x563848b64c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f9544397740 4 client.26255 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563848a47ea0 con 0x563848b62850 2024-06-03T01:22:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.977+0000 7f953a800640 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848b62850 0x563848b64c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.978+0000 7f953a800640 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848b62850 0x563848b64c70 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6929cdf24a7a1211 server_cookie=7c7e6d182e60da87 in_seq=0 out_seq=0 2024-06-03T01:22:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:17.978+0000 7f951be00640 10 client.26255 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f951be00640 1 -- 172.21.15.94:0/2629814603 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563848a47ea0 con 0x563848b62850 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f951be00640 10 client.26255 handle_command_reply: tid=0 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 client.26255 shutdown 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 msgr2=0x563848aeb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aeb900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f95340a45a0 tx=0x7f9534005e30 comp rx=0 tx=0).stop 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 shutdown_connections 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x563848b62850 0x563848b64c70 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f951be00640 0 client.26255 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563848ae6dc0 0x563848aeb900 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563848add5b0 0x563848aeb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 --2- 172.21.15.94:0/2629814603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563848adc040 0x563848aeae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 >> 172.21.15.94:0/2629814603 conn(0x563848aa4f80 msgr2=0x563848ad7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 shutdown_connections 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.007+0000 7f9544397740 1 -- 172.21.15.94:0/2629814603 wait complete. 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9544397740 20 client.26255 trim_cache size 0 max 16384 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:22:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95200a2da0 msgr2=0x7f95200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95200a2da0 0x7f95200a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x563848ad97c0 tx=0x7f9528009b30 comp rx=0 tx=0).stop 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c19ec60 msgr2=0x7f953c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c19ec60 0x7f953c19f110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f953400eb10 tx=0x7f953400c2d0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 shutdown_connections 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f95200a2da0 0x7f95200a5260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f953c19ec60 0x7f953c19f110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f953c108ec0 0x7f953c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 --2- 172.21.15.94:0/2424164322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f953c103df0 0x7f953c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.008+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 >> 172.21.15.94:0/2424164322 conn(0x7f953c0ff690 msgr2=0x7f953c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.009+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 shutdown_connections 2024-06-03T01:22:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:18.009+0000 7f9541400640 1 -- 172.21.15.94:0/2424164322 wait complete. 2024-06-03T01:22:18.138 INFO:tasks.workunit.client.0.smithi094.stdout: 1 265278 2.03 MB/sec execute 505 sec latency 392.893 ms 2024-06-03T01:22:18.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:18 smithi107 ceph-mon[36622]: pgmap v373: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-06-03T01:22:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:18 smithi094 ceph-mon[32119]: pgmap v373: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-06-03T01:22:18.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:18 smithi179 ceph-mon[37041]: pgmap v373: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-06-03T01:22:19.138 INFO:tasks.workunit.client.0.smithi094.stdout: 1 266351 2.04 MB/sec execute 506 sec latency 14.911 ms 2024-06-03T01:22:20.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:19 smithi107 ceph-mon[36622]: pgmap v374: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-03T01:22:20.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:19 smithi094 ceph-mon[32119]: pgmap v374: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-03T01:22:20.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:19 smithi179 ceph-mon[37041]: pgmap v374: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-03T01:22:20.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 266968 2.04 MB/sec execute 507 sec latency 25.480 ms 2024-06-03T01:22:21.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 267752 2.05 MB/sec execute 508 sec latency 117.573 ms 2024-06-03T01:22:22.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 267752 2.04 MB/sec execute 509 sec latency 1117.669 ms 2024-06-03T01:22:22.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:22 smithi107 ceph-mon[36622]: pgmap v375: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 968 KiB/s wr, 105 op/s 2024-06-03T01:22:22.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:22 smithi094 ceph-mon[32119]: pgmap v375: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 968 KiB/s wr, 105 op/s 2024-06-03T01:22:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:22 smithi179 ceph-mon[37041]: pgmap v375: 129 pgs: 129 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 968 KiB/s wr, 105 op/s 2024-06-03T01:22:23.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 268669 2.04 MB/sec execute 510 sec latency 1358.919 ms 2024-06-03T01:22:24.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 269870 2.06 MB/sec execute 511 sec latency 15.776 ms 2024-06-03T01:22:24.406 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:24 smithi179 ceph-mon[37041]: pgmap v376: 129 pgs: 129 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-03T01:22:24.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:24 smithi107 ceph-mon[36622]: pgmap v376: 129 pgs: 129 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-03T01:22:24.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:24 smithi094 ceph-mon[32119]: pgmap v376: 129 pgs: 129 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-03T01:22:25.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.06 MB/sec execute 512 sec latency 287.557 ms 2024-06-03T01:22:26.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.05 MB/sec execute 513 sec latency 1287.670 ms 2024-06-03T01:22:26.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:26 smithi107 ceph-mon[36622]: pgmap v377: 129 pgs: 129 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 997 KiB/s wr, 111 op/s 2024-06-03T01:22:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:26 smithi094 ceph-mon[32119]: pgmap v377: 129 pgs: 129 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 997 KiB/s wr, 111 op/s 2024-06-03T01:22:26.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:26 smithi179 ceph-mon[37041]: pgmap v377: 129 pgs: 129 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 997 KiB/s wr, 111 op/s 2024-06-03T01:22:27.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.05 MB/sec execute 514 sec latency 2287.762 ms 2024-06-03T01:22:28.139 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.05 MB/sec execute 515 sec latency 3287.904 ms 2024-06-03T01:22:28.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:28 smithi107 ceph-mon[36622]: pgmap v378: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-06-03T01:22:28.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:28 smithi094 ceph-mon[32119]: pgmap v378: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-06-03T01:22:28.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:28 smithi179 ceph-mon[37041]: pgmap v378: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-06-03T01:22:29.140 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.04 MB/sec execute 516 sec latency 4288.055 ms 2024-06-03T01:22:30.140 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.04 MB/sec execute 517 sec latency 5288.244 ms 2024-06-03T01:22:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:30 smithi107 ceph-mon[36622]: pgmap v379: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 892 KiB/s wr, 99 op/s 2024-06-03T01:22:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:30 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:30 smithi094 ceph-mon[32119]: pgmap v379: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 892 KiB/s wr, 99 op/s 2024-06-03T01:22:30.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:30 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:30 smithi179 ceph-mon[37041]: pgmap v379: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 892 KiB/s wr, 99 op/s 2024-06-03T01:22:30.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:30 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:31.140 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.03 MB/sec execute 518 sec latency 6288.318 ms 2024-06-03T01:22:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.154+0000 7f9158e00640 1 -- 172.21.15.94:0/3184604120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154103e10 msgr2=0x7f91541041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.154+0000 7f9158e00640 1 --2- 172.21.15.94:0/3184604120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154103e10 0x7f91541041f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9144009900 tx=0x7f914402a3f0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.155+0000 7f9158e00640 1 -- 172.21.15.94:0/3184604120 shutdown_connections 2024-06-03T01:22:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.155+0000 7f9158e00640 1 --2- 172.21.15.94:0/3184604120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9154108ee0 0x7f9154110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.155+0000 7f9158e00640 1 --2- 172.21.15.94:0/3184604120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91541047c0 0x7f9154108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.155+0000 7f9158e00640 1 --2- 172.21.15.94:0/3184604120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154103e10 0x7f91541041f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.155+0000 7f9158e00640 1 -- 172.21.15.94:0/3184604120 >> 172.21.15.94:0/3184604120 conn(0x7f91540ff6d0 msgr2=0x7f9154101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.155+0000 7f9158e00640 1 -- 172.21.15.94:0/3184604120 shutdown_connections 2024-06-03T01:22:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.155+0000 7f9158e00640 1 -- 172.21.15.94:0/3184604120 wait complete. 2024-06-03T01:22:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 Processor -- start 2024-06-03T01:22:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 -- start start 2024-06-03T01:22:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9154103e10 0x7f915419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154108ee0 0x7f915419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f915419ed20 0x7f915419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9154113fe0 con 0x7f9154108ee0 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9154113e60 con 0x7f915419ed20 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.156+0000 7f9158e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9154114160 con 0x7f9154103e10 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9154103e10 0x7f915419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9149400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154108ee0 0x7f915419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9154103e10 0x7f915419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53810/0 (socket says 172.21.15.94:53810) 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 -- 172.21.15.94:0/1765554602 learned_addr learned my addr 172.21.15.94:0/1765554602 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9152800640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f915419ed20 0x7f915419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 -- 172.21.15.94:0/1765554602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f915419ed20 msgr2=0x7f915419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f915419ed20 0x7f915419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 -- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154108ee0 msgr2=0x7f915419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154108ee0 0x7f915419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9151e00640 1 -- 172.21.15.94:0/1765554602 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915419f810 con 0x7f9154103e10 2024-06-03T01:22:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.157+0000 7f9149400640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154108ee0 0x7f915419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.158+0000 7f9151e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9154103e10 0x7f915419a460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9144004770 tx=0x7f9144003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.158+0000 7f914b400640 1 -- 172.21.15.94:0/1765554602 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914403f070 con 0x7f9154103e10 2024-06-03T01:22:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.158+0000 7f914b400640 1 -- 172.21.15.94:0/1765554602 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9144031ca0 con 0x7f9154103e10 2024-06-03T01:22:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.158+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915419faa0 con 0x7f9154103e10 2024-06-03T01:22:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.158+0000 7f914b400640 1 -- 172.21.15.94:0/1765554602 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144035030 con 0x7f9154103e10 2024-06-03T01:22:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.158+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a7200 con 0x7f9154103e10 2024-06-03T01:22:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.161+0000 7f914b400640 1 -- 172.21.15.94:0/1765554602 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9144042070 con 0x7f9154103e10 2024-06-03T01:22:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.161+0000 7f915bda5740 1 Processor -- start 2024-06-03T01:22:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.161+0000 7f915bda5740 1 -- start start 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f915bda5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec037b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f915bda5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec038040 0x557cec0384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f915bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f915bda5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cebf9c420 con 0x557cec0395b0 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f915bda5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cebf43c50 con 0x557cec038040 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f915bda5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557cebf983e0 con 0x557cec042dc0 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f9152800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec037b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f9152800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec037b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58644/0 (socket says 172.21.15.94:58644) 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f9152800640 1 -- 172.21.15.94:0/1139037267 learned_addr learned my addr 172.21.15.94:0/1139037267 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f9149400640 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.162+0000 7f9151e00640 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec038040 0x557cec0384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9152800640 1 -- 172.21.15.94:0/1139037267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 msgr2=0x557cec0451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9152800640 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9152800640 1 -- 172.21.15.94:0/1139037267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec038040 msgr2=0x557cec0384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9152800640 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec038040 0x557cec0384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9152800640 1 -- 172.21.15.94:0/1139037267 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cebcb0210 con 0x557cec0395b0 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9152800640 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec037b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f914c009c20 tx=0x7f914c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cc348e29ec77c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f914b400640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f914b400640 1 -- 172.21.15.94:0/1765554602 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f91440efdf0 con 0x7f9154103e10 2024-06-03T01:22:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9143400640 1 -- 172.21.15.94:0/1139037267 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c01e070 con 0x557cec0395b0 2024-06-03T01:22:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9151e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9143400640 1 -- 172.21.15.94:0/1139037267 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f914c019b10 con 0x557cec0395b0 2024-06-03T01:22:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.163+0000 7f9143400640 1 -- 172.21.15.94:0/1139037267 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c00ea10 con 0x557cec0395b0 2024-06-03T01:22:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 -- 172.21.15.94:0/1139037267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 msgr2=0x557cec037b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec037b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f914c009c20 tx=0x7f914c011810 comp rx=0 tx=0).stop 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f9151e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91440ed230 tx=0x7f91440f3940 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 -- 172.21.15.94:0/1139037267 shutdown_connections 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec038040 0x557cec0384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 --2- 172.21.15.94:0/1139037267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec037b00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 -- 172.21.15.94:0/1139037267 >> 172.21.15.94:0/1139037267 conn(0x557cec000f80 msgr2=0x557cec034630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 -- 172.21.15.94:0/1139037267 shutdown_connections 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.164+0000 7f915bda5740 1 -- 172.21.15.94:0/1139037267 wait complete. 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 -- start start 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 Processor -- start 2024-06-03T01:22:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec03fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec040350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec040890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cec045b90 con 0x557cec038040 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557cebf570b0 con 0x557cec0395b0 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.165+0000 7f915bda5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557cec045d10 con 0x557cec042dc0 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f915bda5740 10 client.0 _ll_register_callbacks cb 0x557cebf42f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec03fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec03fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58652/0 (socket says 172.21.15.94:58652) 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 -- 172.21.15.94:0/3366463808 learned_addr learned my addr 172.21.15.94:0/3366463808 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9149400640 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec040350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 -- 172.21.15.94:0/3366463808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 msgr2=0x557cec040890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec040890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 -- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 msgr2=0x557cec040350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec040350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 -- 172.21.15.94:0/3366463808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cebd7c620 con 0x557cec038040 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9152800640 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec03fe10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f914c00a010 tx=0x7f914c00e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84454123ac88f0d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.166+0000 7f9137e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.167+0000 7f9137e00640 1 -- 172.21.15.94:0/3366463808 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c02c060 con 0x557cec038040 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.167+0000 7f9137e00640 1 -- 172.21.15.94:0/3366463808 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f914c004030 con 0x557cec038040 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.167+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f914c019b10 con 0x557cec038040 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.167+0000 7f9137e00640 1 -- 172.21.15.94:0/3366463808 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f914c033050 con 0x557cec038040 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.167+0000 7f915bda5740 10 client.16152 fetch_fsmap learned FSMap version 57 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.167+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x557cec051f30 con 0x557cec038040 2024-06-03T01:22:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.167+0000 7f9137e00640 1 -- 172.21.15.94:0/3366463808 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f914c019b10 con 0x557cec038040 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f9137e00640 1 -- 172.21.15.94:0/3366463808 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 57) ==== 8394+0+0 (secure 0 0 0) 0x7f914c024070 con 0x557cec038040 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f915bda5740 10 client.16152 fetch_fsmap finished waiting for FSMap version 57 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f915bda5740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f915bda5740 20 client.16152 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f915bda5740 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0532e0 0x557cec0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f915bda5740 4 client.16152 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cebfa9b50 con 0x557cec0532e0 2024-06-03T01:22:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.168+0000 7f9149400640 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0532e0 0x557cec0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.169+0000 7f9137e00640 10 client.16152 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.169+0000 7f9149400640 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0532e0 0x557cec0556d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d1a07dd7b8e6718 server_cookie=34d7169356babaa3 in_seq=0 out_seq=0 2024-06-03T01:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f9137e00640 1 -- 172.21.15.94:0/3366463808 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557cebfa9b50 con 0x557cec0532e0 2024-06-03T01:22:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f9137e00640 10 client.16152 handle_command_reply: tid=0 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 client.16152 shutdown 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 msgr2=0x557cec03fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec03fe10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f914c00a010 tx=0x7f914c00e5f0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 shutdown_connections 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec040890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0532e0 0x557cec0556d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec040350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 --2- 172.21.15.94:0/3366463808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec03fe10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f9137e00640 0 client.16152 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 >> 172.21.15.94:0/3366463808 conn(0x557cec000f80 msgr2=0x557cec03a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 shutdown_connections 2024-06-03T01:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 1 -- 172.21.15.94:0/3366463808 wait complete. 2024-06-03T01:22:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.170+0000 7f915bda5740 20 client.16152 trim_cache size 0 max 16384 2024-06-03T01:22:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 Processor -- start 2024-06-03T01:22:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 -- start start 2024-06-03T01:22:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec038040 0x557cec05adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec05b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec042dc0 0x557cec039ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b10d0 con 0x557cec0395b0 2024-06-03T01:22:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b2d00 con 0x557cec042dc0 2024-06-03T01:22:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f915bda5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b17d0 con 0x557cec038040 2024-06-03T01:22:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f9149400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec05b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f9149400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec05b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58658/0 (socket says 172.21.15.94:58658) 2024-06-03T01:22:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.187+0000 7f9149400640 1 -- 172.21.15.94:0/4242937312 learned_addr learned my addr 172.21.15.94:0/4242937312 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9152800640 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec038040 0x557cec05adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9152800640 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec042dc0 0x557cec039ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9149400640 1 -- 172.21.15.94:0/4242937312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec038040 msgr2=0x557cec05adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9149400640 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec038040 0x557cec05adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9149400640 1 -- 172.21.15.94:0/4242937312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec042dc0 msgr2=0x557cec039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9149400640 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec042dc0 0x557cec039ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9149400640 1 -- 172.21.15.94:0/4242937312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cec038b10 con 0x557cec0395b0 2024-06-03T01:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9152800640 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec042dc0 0x557cec039ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9149400640 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec05b300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f913c00f840 tx=0x7f913c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1af4bbe1071e8f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:31.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9142a00640 1 -- 172.21.15.94:0/4242937312 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00fe20 con 0x557cec0395b0 2024-06-03T01:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9142a00640 1 -- 172.21.15.94:0/4242937312 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f913c010430 con 0x557cec0395b0 2024-06-03T01:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.188+0000 7f9142a00640 1 -- 172.21.15.94:0/4242937312 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c01e680 con 0x557cec0395b0 2024-06-03T01:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 -- 172.21.15.94:0/4242937312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 msgr2=0x557cec05b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec05b300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f913c00f840 tx=0x7f913c009500 comp rx=0 tx=0).stop 2024-06-03T01:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 -- 172.21.15.94:0/4242937312 shutdown_connections 2024-06-03T01:22:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec042dc0 0x557cec039ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec0395b0 0x557cec05b300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 --2- 172.21.15.94:0/4242937312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec038040 0x557cec05adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 -- 172.21.15.94:0/4242937312 >> 172.21.15.94:0/4242937312 conn(0x557cec000f80 msgr2=0x557cec03b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 -- 172.21.15.94:0/4242937312 shutdown_connections 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.189+0000 7f915bda5740 1 -- 172.21.15.94:0/4242937312 wait complete. 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 -- start start 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 Processor -- start 2024-06-03T01:22:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec046710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec046c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b2600 con 0x557cec038040 2024-06-03T01:22:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91541b1f30 con 0x557cec0395b0 2024-06-03T01:22:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f91541af0d0 con 0x557cec042dc0 2024-06-03T01:22:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.190+0000 7f915bda5740 10 client.0 _ll_register_callbacks cb 0x7f914c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec046710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9149400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec046c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec046710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58672/0 (socket says 172.21.15.94:58672) 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9149400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec046c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:44290/0 (socket says 172.21.15.94:44290) 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 -- 172.21.15.94:0/3216682359 learned_addr learned my addr 172.21.15.94:0/3216682359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 -- 172.21.15.94:0/3216682359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 msgr2=0x557cec0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 -- 172.21.15.94:0/3216682359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 msgr2=0x557cec046c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec046c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 -- 172.21.15.94:0/3216682359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cec0b6390 con 0x557cec038040 2024-06-03T01:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9152800640 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec046710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f914c00eb10 tx=0x7f914c00e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef859f049d1db81 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9137e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-06-03T01:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9137e00640 1 -- 172.21.15.94:0/3216682359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c02c070 con 0x557cec038040 2024-06-03T01:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f9137e00640 1 -- 172.21.15.94:0/3216682359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f914c005240 con 0x557cec038040 2024-06-03T01:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.191+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f913c010430 con 0x557cec038040 2024-06-03T01:22:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f9137e00640 1 -- 172.21.15.94:0/3216682359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f914c01e040 con 0x557cec038040 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 10 client.16158 fetch_fsmap learned FSMap version 57 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x557cec0bd6c0 con 0x557cec038040 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f9137e00640 1 -- 172.21.15.94:0/3216682359 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f913c010430 con 0x557cec038040 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f9137e00640 1 -- 172.21.15.94:0/3216682359 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 57) ==== 8394+0+0 (secure 0 0 0) 0x7f914c024040 con 0x557cec038040 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 10 client.16158 fetch_fsmap finished waiting for FSMap version 57 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 20 client.16158 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0bdde0 0x557cec0c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 4 client.16158 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.192+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cebfa3ea0 con 0x557cec0bdde0 2024-06-03T01:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.193+0000 7f9149400640 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0bdde0 0x557cec0c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.193+0000 7f9149400640 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0bdde0 0x557cec0c0200 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ac8dca4e733c1 server_cookie=c9928da6c2dc84e1 in_seq=0 out_seq=0 2024-06-03T01:22:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.193+0000 7f9137e00640 10 client.16158 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f9137e00640 1 -- 172.21.15.94:0/3216682359 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557cebfa3ea0 con 0x557cec0bdde0 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f9137e00640 10 client.16158 handle_command_reply: tid=0 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 client.16158 shutdown 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 msgr2=0x557cec046710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec046710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f914c00eb10 tx=0x7f914c00e5f0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 shutdown_connections 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557cec042dc0 0x557cec0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x557cec0bdde0 0x557cec0c0200 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557cec0395b0 0x557cec046c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 --2- 172.21.15.94:0/3216682359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cec038040 0x557cec046710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 >> 172.21.15.94:0/3216682359 conn(0x557cec000f80 msgr2=0x557cec03fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.222+0000 7f9137e00640 0 client.16158 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.223+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 shutdown_connections 2024-06-03T01:22:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.223+0000 7f915bda5740 1 -- 172.21.15.94:0/3216682359 wait complete. 2024-06-03T01:22:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.223+0000 7f915bda5740 20 client.16158 trim_cache size 0 max 16384 2024-06-03T01:22:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:22:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:22:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:22:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 msgr2=0x7f91380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91440ed230 tx=0x7f91440f3940 comp rx=0 tx=0).stop 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9154103e10 msgr2=0x7f915419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9154103e10 0x7f915419a460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9144004770 tx=0x7f9144003c10 comp rx=0 tx=0).stop 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 shutdown_connections 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f91380a2da0 0x7f91380a5260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f915419ed20 0x7f915419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9154108ee0 0x7f915419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 --2- 172.21.15.94:0/1765554602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9154103e10 0x7f915419a460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 >> 172.21.15.94:0/1765554602 conn(0x7f91540ff6d0 msgr2=0x7f91541019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.224+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 shutdown_connections 2024-06-03T01:22:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:31.225+0000 7f9158e00640 1 -- 172.21.15.94:0/1765554602 wait complete. 2024-06-03T01:22:32.140 INFO:tasks.workunit.client.0.smithi094.stdout: 1 270204 2.03 MB/sec execute 519 sec latency 7288.439 ms 2024-06-03T01:22:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:32 smithi107 ceph-mon[36622]: pgmap v380: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 892 KiB/s wr, 99 op/s 2024-06-03T01:22:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:32 smithi094 ceph-mon[32119]: pgmap v380: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 892 KiB/s wr, 99 op/s 2024-06-03T01:22:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:32 smithi179 ceph-mon[37041]: pgmap v380: 129 pgs: 129 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 892 KiB/s wr, 99 op/s 2024-06-03T01:22:33.140 INFO:tasks.workunit.client.0.smithi094.stdout: 1 271027 2.04 MB/sec execute 520 sec latency 7529.488 ms 2024-06-03T01:22:34.140 INFO:tasks.workunit.client.0.smithi094.stdout: 1 271294 2.03 MB/sec execute 521 sec latency 724.149 ms 2024-06-03T01:22:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:34 smithi107 ceph-mon[36622]: pgmap v381: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 942 KiB/s wr, 118 op/s 2024-06-03T01:22:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:34 smithi094 ceph-mon[32119]: pgmap v381: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 942 KiB/s wr, 118 op/s 2024-06-03T01:22:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:34 smithi179 ceph-mon[37041]: pgmap v381: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 942 KiB/s wr, 118 op/s 2024-06-03T01:22:35.140 INFO:tasks.workunit.client.0.smithi094.stdout: 1 271294 2.03 MB/sec execute 522 sec latency 1724.310 ms 2024-06-03T01:22:36.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 271294 2.02 MB/sec execute 523 sec latency 2724.447 ms 2024-06-03T01:22:36.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:36 smithi107 ceph-mon[36622]: pgmap v382: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 581 KiB/s wr, 61 op/s 2024-06-03T01:22:36.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:36 smithi094 ceph-mon[32119]: pgmap v382: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 581 KiB/s wr, 61 op/s 2024-06-03T01:22:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:36 smithi179 ceph-mon[37041]: pgmap v382: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 581 KiB/s wr, 61 op/s 2024-06-03T01:22:37.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 271294 2.02 MB/sec execute 524 sec latency 3724.562 ms 2024-06-03T01:22:38.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 272369 2.03 MB/sec execute 525 sec latency 3968.140 ms 2024-06-03T01:22:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:38 smithi107 ceph-mon[36622]: pgmap v383: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 722 KiB/s wr, 84 op/s 2024-06-03T01:22:38.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:38 smithi094 ceph-mon[32119]: pgmap v383: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 722 KiB/s wr, 84 op/s 2024-06-03T01:22:38.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:38 smithi179 ceph-mon[37041]: pgmap v383: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 722 KiB/s wr, 84 op/s 2024-06-03T01:22:39.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 273435 2.04 MB/sec execute 526 sec latency 16.573 ms 2024-06-03T01:22:40.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 274065 2.04 MB/sec execute 527 sec latency 30.923 ms 2024-06-03T01:22:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:40 smithi107 ceph-mon[36622]: pgmap v384: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 191 KiB/s wr, 42 op/s 2024-06-03T01:22:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:40 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:40 smithi094 ceph-mon[32119]: pgmap v384: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 191 KiB/s wr, 42 op/s 2024-06-03T01:22:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:40 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:40.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:40 smithi179 ceph-mon[37041]: pgmap v384: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 191 KiB/s wr, 42 op/s 2024-06-03T01:22:40.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:40 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:41.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 274843 2.04 MB/sec execute 528 sec latency 134.193 ms 2024-06-03T01:22:42.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 274843 2.04 MB/sec execute 529 sec latency 1134.340 ms 2024-06-03T01:22:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:42 smithi107 ceph-mon[36622]: pgmap v385: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 191 KiB/s wr, 42 op/s 2024-06-03T01:22:42.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:42 smithi094 ceph-mon[32119]: pgmap v385: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 191 KiB/s wr, 42 op/s 2024-06-03T01:22:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:42 smithi179 ceph-mon[37041]: pgmap v385: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 191 KiB/s wr, 42 op/s 2024-06-03T01:22:43.141 INFO:tasks.workunit.client.0.smithi094.stdout: 1 275908 2.04 MB/sec execute 530 sec latency 1372.442 ms 2024-06-03T01:22:44.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 276969 2.05 MB/sec execute 531 sec latency 24.016 ms 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 -- 172.21.15.94:0/3937241149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4102370 msgr2=0x7f66e410e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 --2- 172.21.15.94:0/3937241149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4102370 0x7f66e410e880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009900 tx=0x7f66dc02a410 comp rx=0 tx=0).stop 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 -- 172.21.15.94:0/3937241149 shutdown_connections 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 --2- 172.21.15.94:0/3937241149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4102370 0x7f66e410e880 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 --2- 172.21.15.94:0/3937241149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e41019b0 0x7f66e4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 --2- 172.21.15.94:0/3937241149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e41079c0 0x7f66e4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 -- 172.21.15.94:0/3937241149 >> 172.21.15.94:0/3937241149 conn(0x7f66e4078e30 msgr2=0x7f66e4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 -- 172.21.15.94:0/3937241149 shutdown_connections 2024-06-03T01:22:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.370+0000 7f66eae00640 1 -- 172.21.15.94:0/3937241149 wait complete. 2024-06-03T01:22:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.371+0000 7f66eae00640 1 Processor -- start 2024-06-03T01:22:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66eae00640 1 -- start start 2024-06-03T01:22:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66eae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e41019b0 0x7f66e419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66eae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e4102370 0x7f66e419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66eae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e41079c0 0x7f66e419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66eae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111e20 con 0x7f66e41019b0 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66eae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111ca0 con 0x7f66e41079c0 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66eae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111fa0 con 0x7f66e4102370 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66e8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e41079c0 0x7f66e419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66e8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e41079c0 0x7f66e419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:44614/0 (socket says 172.21.15.94:44614) 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66e8e00640 1 -- 172.21.15.94:0/1466456153 learned_addr learned my addr 172.21.15.94:0/1466456153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66db400640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e4102370 0x7f66e419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.372+0000 7f66e3e00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e41019b0 0x7f66e419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e8e00640 1 -- 172.21.15.94:0/1466456153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e4102370 msgr2=0x7f66e419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e8e00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e4102370 0x7f66e419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e8e00640 1 -- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e41019b0 msgr2=0x7f66e419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e8e00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e41019b0 0x7f66e419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e8e00640 1 -- 172.21.15.94:0/1466456153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e419f350 con 0x7f66e41079c0 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66db400640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e4102370 0x7f66e419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e8e00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e41079c0 0x7f66e419ec20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f66dc02a930 tx=0x7f66dc003ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e1600640 1 -- 172.21.15.94:0/1466456153 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc03f070 con 0x7f66e41079c0 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e1600640 1 -- 172.21.15.94:0/1466456153 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66dc031c20 con 0x7f66e41079c0 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e419f5e0 con 0x7f66e41079c0 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.373+0000 7f66e1600640 1 -- 172.21.15.94:0/1466456153 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc035030 con 0x7f66e41079c0 2024-06-03T01:22:44.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.374+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e41a6ed0 con 0x7f66e41079c0 2024-06-03T01:22:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.377+0000 7f66e1600640 1 -- 172.21.15.94:0/1466456153 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f66dc042070 con 0x7f66e41079c0 2024-06-03T01:22:44.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.377+0000 7f66eddbf740 1 Processor -- start 2024-06-03T01:22:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.377+0000 7f66eddbf740 1 -- start start 2024-06-03T01:22:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad45b0 0x561199ad2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ad34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199adddc0 0x561199ae01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66eddbf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561199a37420 con 0x561199adddc0 2024-06-03T01:22:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66eddbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5611999dec50 con 0x561199ad3040 2024-06-03T01:22:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66eddbf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561199a333e0 con 0x561199ad45b0 2024-06-03T01:22:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ad34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ad34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:44616/0 (socket says 172.21.15.94:44616) 2024-06-03T01:22:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66e3e00640 1 -- 172.21.15.94:0/2387503253 learned_addr learned my addr 172.21.15.94:0/2387503253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66db400640 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad45b0 0x561199ad2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.378+0000 7f66db400640 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199adddc0 0x561199ae01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e3e00640 1 -- 172.21.15.94:0/2387503253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad45b0 msgr2=0x561199ad2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e3e00640 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad45b0 0x561199ad2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e3e00640 1 -- 172.21.15.94:0/2387503253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199adddc0 msgr2=0x561199ae01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e3e00640 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199adddc0 0x561199ae01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e3e00640 1 -- 172.21.15.94:0/2387503253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119974b210 con 0x561199ad3040 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e1600640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f66c80a2e20 0x7f66c80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e1600640 1 -- 172.21.15.94:0/1466456153 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f66dc0efcd0 con 0x7f66e41079c0 2024-06-03T01:22:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66e3e00640 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ad34c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f66d4011440 tx=0x7f66d4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df66c5ba2b0821c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66db400640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f66c80a2e20 0x7f66c80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66d9600640 1 -- 172.21.15.94:0/2387503253 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4003c30 con 0x561199ad3040 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66d9600640 1 -- 172.21.15.94:0/2387503253 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d4003dd0 con 0x561199ad3040 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.379+0000 7f66d9600640 1 -- 172.21.15.94:0/2387503253 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d400fb90 con 0x561199ad3040 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 -- 172.21.15.94:0/2387503253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 msgr2=0x561199ad34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ad34c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f66d4011440 tx=0x7f66d4011900 comp rx=0 tx=0).stop 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66db400640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f66c80a2e20 0x7f66c80a52e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x561199ad07c0 tx=0x7f66cc005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 -- 172.21.15.94:0/2387503253 shutdown_connections 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199adddc0 0x561199ae01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ad34c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 --2- 172.21.15.94:0/2387503253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad45b0 0x561199ad2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 -- 172.21.15.94:0/2387503253 >> 172.21.15.94:0/2387503253 conn(0x561199a9bf80 msgr2=0x561199acf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 -- 172.21.15.94:0/2387503253 shutdown_connections 2024-06-03T01:22:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.380+0000 7f66eddbf740 1 -- 172.21.15.94:0/2387503253 wait complete. 2024-06-03T01:22:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.381+0000 7f66eddbf740 1 -- start start 2024-06-03T01:22:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.381+0000 7f66eddbf740 1 Processor -- start 2024-06-03T01:22:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.381+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199adadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.381+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199adb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.381+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199adb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66eddbf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561199ae0b90 con 0x561199ad45b0 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66eddbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5611999f20b0 con 0x561199adddc0 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66eddbf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561199ae0d10 con 0x561199ad3040 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66eddbf740 10 client.0 _ll_register_callbacks cb 0x5611999ddf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199adadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199adb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199adadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43462/0 (socket says 172.21.15.94:43462) 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 -- 172.21.15.94:0/3334908332 learned_addr learned my addr 172.21.15.94:0/3334908332 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e8e00640 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199adb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 -- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 msgr2=0x561199adb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199adb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 -- 172.21.15.94:0/3334908332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 msgr2=0x561199adb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199adb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.382+0000 7f66e3e00640 1 -- 172.21.15.94:0/3334908332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561199817620 con 0x561199ad3040 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66e3e00640 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199adadb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66d40099d0 tx=0x7f66d4004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1d121ef4ad6a17 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66d2000640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66d2000640 1 -- 172.21.15.94:0/3334908332 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4028020 con 0x561199ad3040 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d4003dd0 con 0x561199ad3040 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66d2000640 1 -- 172.21.15.94:0/3334908332 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f66d400fd00 con 0x561199ad3040 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66d2000640 1 -- 172.21.15.94:0/3334908332 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f66d4033050 con 0x561199ad3040 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66eddbf740 10 client.26296 fetch_fsmap learned FSMap version 59 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561199aecf00 con 0x561199ad3040 2024-06-03T01:22:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.383+0000 7f66d2000640 1 -- 172.21.15.94:0/3334908332 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d4003dd0 con 0x561199ad3040 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66d2000640 1 -- 172.21.15.94:0/3334908332 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 59) ==== 8394+0+0 (secure 0 0 0) 0x7f66d40050d0 con 0x561199ad3040 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66eddbf740 10 client.26296 fetch_fsmap finished waiting for FSMap version 59 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66eddbf740 10 client.26296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66eddbf740 20 client.26296 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199aee360 0x561199af0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66eddbf740 4 client.26296 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561199a44b50 con 0x561199aee360 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.384+0000 7f66db400640 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199aee360 0x561199af0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.385+0000 7f66db400640 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199aee360 0x561199af0750 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73734a04de35b31d server_cookie=76e92fedb2f06704 in_seq=0 out_seq=0 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.385+0000 7f66d2000640 10 client.26296 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.385+0000 7f66d2000640 1 -- 172.21.15.94:0/3334908332 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561199a44b50 con 0x561199aee360 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66d2000640 10 client.26296 handle_command_reply: tid=0 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 client.26296 shutdown 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 msgr2=0x561199adadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199adadb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66d40099d0 tx=0x7f66d4004cf0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 shutdown_connections 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199aee360 0x561199af0750 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199adb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199adb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3334908332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199adadb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66d2000640 0 client.26296 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 >> 172.21.15.94:0/3334908332 conn(0x561199a9bf80 msgr2=0x561199ad54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 shutdown_connections 2024-06-03T01:22:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 1 -- 172.21.15.94:0/3334908332 wait complete. 2024-06-03T01:22:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.386+0000 7f66eddbf740 20 client.26296 trim_cache size 0 max 16384 2024-06-03T01:22:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 Processor -- start 2024-06-03T01:22:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 -- start start 2024-06-03T01:22:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ace690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199af5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199adddc0 0x561199af64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b1020 con 0x561199ad45b0 2024-06-03T01:22:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b2c50 con 0x561199ad3040 2024-06-03T01:22:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.406+0000 7f66eddbf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b1750 con 0x561199adddc0 2024-06-03T01:22:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199adddc0 0x561199af64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199adddc0 0x561199af64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43470/0 (socket says 172.21.15.94:43470) 2024-06-03T01:22:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 -- 172.21.15.94:0/715315199 learned_addr learned my addr 172.21.15.94:0/715315199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e3e00640 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ace690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66db400640 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199af5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 -- 172.21.15.94:0/715315199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 msgr2=0x561199ace690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ace690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 -- 172.21.15.94:0/715315199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 msgr2=0x561199af5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199af5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 -- 172.21.15.94:0/715315199 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561199ad9d60 con 0x561199adddc0 2024-06-03T01:22:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.407+0000 7f66e8e00640 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199adddc0 0x561199af64c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0bbb80 tx=0x7f66dc03b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d367b3c32d1afbc server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.408+0000 7f66d8c00640 1 -- 172.21.15.94:0/715315199 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc049070 con 0x561199adddc0 2024-06-03T01:22:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.408+0000 7f66d8c00640 1 -- 172.21.15.94:0/715315199 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66dc0c0180 con 0x561199adddc0 2024-06-03T01:22:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.408+0000 7f66d8c00640 1 -- 172.21.15.94:0/715315199 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0bc0e0 con 0x561199adddc0 2024-06-03T01:22:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.408+0000 7f66eddbf740 1 -- 172.21.15.94:0/715315199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199adddc0 msgr2=0x561199af64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.408+0000 7f66eddbf740 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199adddc0 0x561199af64c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0bbb80 tx=0x7f66dc03b670 comp rx=0 tx=0).stop 2024-06-03T01:22:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 -- 172.21.15.94:0/715315199 shutdown_connections 2024-06-03T01:22:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199adddc0 0x561199af64c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199af5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 --2- 172.21.15.94:0/715315199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199ad3040 0x561199ace690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 -- 172.21.15.94:0/715315199 >> 172.21.15.94:0/715315199 conn(0x561199a9bf80 msgr2=0x561199acdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 -- 172.21.15.94:0/715315199 shutdown_connections 2024-06-03T01:22:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 -- 172.21.15.94:0/715315199 wait complete. 2024-06-03T01:22:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 -- start start 2024-06-03T01:22:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.409+0000 7f66eddbf740 1 Processor -- start 2024-06-03T01:22:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199ae65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199ad7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66eddbf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199ad7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66eddbf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b2580 con 0x561199ad45b0 2024-06-03T01:22:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66eddbf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b1eb0 con 0x561199adddc0 2024-06-03T01:22:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66eddbf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41af020 con 0x561199ad3040 2024-06-03T01:22:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66eddbf740 10 client.0 _ll_register_callbacks cb 0x7f66d4009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199ae65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66e8e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199ad7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199ae65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43472/0 (socket says 172.21.15.94:43472) 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66e3e00640 1 -- 172.21.15.94:0/3936176040 learned_addr learned my addr 172.21.15.94:0/3936176040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.410+0000 7f66db400640 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199ad7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66e8e00640 1 -- 172.21.15.94:0/3936176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 msgr2=0x561199ae65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66e8e00640 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199ae65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66e8e00640 1 -- 172.21.15.94:0/3936176040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 msgr2=0x561199ad7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66e8e00640 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199ad7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66e8e00640 1 -- 172.21.15.94:0/3936176040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561199ad8510 con 0x561199adddc0 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66e3e00640 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199ae65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66e8e00640 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199ad7ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0bbb80 tx=0x7f66dc005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebecd775e4bb6e3f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66d2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66d2000640 1 -- 172.21.15.94:0/3936176040 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc09e020 con 0x561199adddc0 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66d2000640 1 -- 172.21.15.94:0/3936176040 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7f66dc0b84f0 con 0x561199adddc0 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66dc0c0180 con 0x561199adddc0 2024-06-03T01:22:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66d2000640 1 -- 172.21.15.94:0/3936176040 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f66dc049040 con 0x561199adddc0 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66eddbf740 10 client.26288 fetch_fsmap learned FSMap version 59 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.411+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x561199b58dd0 con 0x561199adddc0 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66d2000640 1 -- 172.21.15.94:0/3936176040 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66dc0c0180 con 0x561199adddc0 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66d2000640 1 -- 172.21.15.94:0/3936176040 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 59) ==== 8394+0+0 (secure 0 0 0) 0x7f66dc0a1080 con 0x561199adddc0 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66eddbf740 10 client.26288 fetch_fsmap finished waiting for FSMap version 59 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66eddbf740 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66eddbf740 20 client.26288 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199b59280 0x561199b5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66eddbf740 4 client.26288 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:22:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561199a3eea0 con 0x561199b59280 2024-06-03T01:22:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.412+0000 7f66e3e00640 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199b59280 0x561199b5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.413+0000 7f66e3e00640 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199b59280 0x561199b5b6a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa41574dc69d766 server_cookie=c6a43ec0779a0164 in_seq=0 out_seq=0 2024-06-03T01:22:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.413+0000 7f66d2000640 10 client.26288 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.445+0000 7f66d2000640 1 -- 172.21.15.94:0/3936176040 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561199a3eea0 con 0x561199b59280 2024-06-03T01:22:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.445+0000 7f66d2000640 10 client.26288 handle_command_reply: tid=0 2024-06-03T01:22:44.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.445+0000 7f66eddbf740 1 client.26288 shutdown 2024-06-03T01:22:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.445+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 msgr2=0x561199ad7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.445+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199ad7ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0bbb80 tx=0x7f66dc005c90 comp rx=0 tx=0).stop 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 shutdown_connections 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561199b59280 0x561199b5b6a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561199adddc0 0x561199ad7ed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561199ad45b0 0x561199ad7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 --2- 172.21.15.94:0/3936176040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561199ad3040 0x561199ae65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 >> 172.21.15.94:0/3936176040 conn(0x561199a9bf80 msgr2=0x561199ace810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66d2000640 0 client.26288 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 shutdown_connections 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 1 -- 172.21.15.94:0/3936176040 wait complete. 2024-06-03T01:22:44.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.446+0000 7f66eddbf740 20 client.26288 trim_cache size 0 max 16384 2024-06-03T01:22:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:22:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:22:44.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:22:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f66c80a2e20 msgr2=0x7f66c80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f66c80a2e20 0x7f66c80a52e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x561199ad07c0 tx=0x7f66cc005f90 comp rx=0 tx=0).stop 2024-06-03T01:22:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e41079c0 msgr2=0x7f66e419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e41079c0 0x7f66e419ec20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f66dc02a930 tx=0x7f66dc003ae0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 shutdown_connections 2024-06-03T01:22:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f66c80a2e20 0x7f66c80a52e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e41079c0 0x7f66e419ec20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e4102370 0x7f66e419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 --2- 172.21.15.94:0/1466456153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e41019b0 0x7f66e419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 >> 172.21.15.94:0/1466456153 conn(0x7f66e4078e30 msgr2=0x7f66e4074220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 shutdown_connections 2024-06-03T01:22:44.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:44.447+0000 7f66eae00640 1 -- 172.21.15.94:0/1466456153 wait complete. 2024-06-03T01:22:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:44 smithi107 ceph-mon[36622]: pgmap v386: 129 pgs: 129 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 764 KiB/s wr, 97 op/s 2024-06-03T01:22:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:44 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:44 smithi094 ceph-mon[32119]: pgmap v386: 129 pgs: 129 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 764 KiB/s wr, 97 op/s 2024-06-03T01:22:44.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:44 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:44 smithi179 ceph-mon[37041]: pgmap v386: 129 pgs: 129 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 764 KiB/s wr, 97 op/s 2024-06-03T01:22:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:44 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:45.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 277608 2.05 MB/sec execute 532 sec latency 19.801 ms 2024-06-03T01:22:46.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 278385 2.06 MB/sec execute 533 sec latency 269.529 ms 2024-06-03T01:22:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:46 smithi107 ceph-mon[36622]: pgmap v387: 129 pgs: 129 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 714 KiB/s wr, 77 op/s 2024-06-03T01:22:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:46 smithi094 ceph-mon[32119]: pgmap v387: 129 pgs: 129 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 714 KiB/s wr, 77 op/s 2024-06-03T01:22:46.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:46 smithi179 ceph-mon[37041]: pgmap v387: 129 pgs: 129 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 714 KiB/s wr, 77 op/s 2024-06-03T01:22:47.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 278385 2.05 MB/sec execute 534 sec latency 1269.628 ms 2024-06-03T01:22:47.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:47 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:47.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:47 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:47.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:47 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:22:48.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 279567 2.06 MB/sec execute 535 sec latency 1509.486 ms 2024-06-03T01:22:48.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:48 smithi107 ceph-mon[36622]: pgmap v388: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-06-03T01:22:48.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:48 smithi094 ceph-mon[32119]: pgmap v388: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-06-03T01:22:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:48 smithi179 ceph-mon[37041]: pgmap v388: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-06-03T01:22:49.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 280626 2.07 MB/sec execute 536 sec latency 16.270 ms 2024-06-03T01:22:50.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 281463 2.07 MB/sec execute 537 sec latency 13.114 ms 2024-06-03T01:22:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:50 smithi107 ceph-mon[36622]: pgmap v389: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-06-03T01:22:50.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:50 smithi094 ceph-mon[32119]: pgmap v389: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-06-03T01:22:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:50 smithi179 ceph-mon[37041]: pgmap v389: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-06-03T01:22:51.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 281934 2.07 MB/sec execute 538 sec latency 615.268 ms 2024-06-03T01:22:51.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:51 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:51.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:51 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:51.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:51 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:22:52.142 INFO:tasks.workunit.client.0.smithi094.stdout: 1 281934 2.07 MB/sec execute 539 sec latency 1615.389 ms 2024-06-03T01:22:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:52 smithi107 ceph-mon[36622]: pgmap v390: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-06-03T01:22:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:52 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:52 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:52 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:22:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:52 smithi094 ceph-mon[32119]: pgmap v390: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-06-03T01:22:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:52 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:52 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:52 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:22:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:52 smithi179 ceph-mon[37041]: pgmap v390: 129 pgs: 129 active+clean; 111 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-06-03T01:22:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:52 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:22:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:52 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:22:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:52 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:22:53.143 INFO:tasks.workunit.client.0.smithi094.stdout: 1 283119 2.08 MB/sec execute 540 sec latency 1860.727 ms 2024-06-03T01:22:54.143 INFO:tasks.workunit.client.0.smithi094.stdout: 1 284171 2.09 MB/sec execute 541 sec latency 17.236 ms 2024-06-03T01:22:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:54 smithi107 ceph-mon[36622]: pgmap v391: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-06-03T01:22:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:54 smithi094 ceph-mon[32119]: pgmap v391: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-06-03T01:22:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:54 smithi179 ceph-mon[37041]: pgmap v391: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-06-03T01:22:55.143 INFO:tasks.workunit.client.0.smithi094.stdout: 1 284967 2.09 MB/sec execute 542 sec latency 32.765 ms 2024-06-03T01:22:56.143 INFO:tasks.workunit.client.0.smithi094.stdout: 1 285476 2.09 MB/sec execute 543 sec latency 574.370 ms 2024-06-03T01:22:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:56 smithi107 ceph-mon[36622]: pgmap v392: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-06-03T01:22:56.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:56 smithi094 ceph-mon[32119]: pgmap v392: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-06-03T01:22:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:56 smithi179 ceph-mon[37041]: pgmap v392: 129 pgs: 129 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-06-03T01:22:57.143 INFO:tasks.workunit.client.0.smithi094.stdout: 1 285476 2.09 MB/sec execute 544 sec latency 1574.530 ms 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 -- 172.21.15.94:0/3889580711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901039e0 msgr2=0x7fc590103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 --2- 172.21.15.94:0/3889580711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901039e0 0x7fc590103e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc578009900 tx=0x7fc57802a3f0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 -- 172.21.15.94:0/3889580711 shutdown_connections 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 --2- 172.21.15.94:0/3889580711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 0x7fc5901108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 --2- 172.21.15.94:0/3889580711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901039e0 0x7fc590103e60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 --2- 172.21.15.94:0/3889580711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901099f0 0x7fc590109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 -- 172.21.15.94:0/3889580711 >> 172.21.15.94:0/3889580711 conn(0x7fc5900ff4b0 msgr2=0x7fc5901018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.591+0000 7fc58f200640 1 -- 172.21.15.94:0/3889580711 shutdown_connections 2024-06-03T01:22:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.592+0000 7fc58f200640 1 -- 172.21.15.94:0/3889580711 wait complete. 2024-06-03T01:22:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.592+0000 7fc58f200640 1 Processor -- start 2024-06-03T01:22:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58f200640 1 -- start start 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901039e0 0x7fc59019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 0x7fc59019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901099f0 0x7fc59019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc590113e00 con 0x7fc5901039e0 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc590113c80 con 0x7fc5901099f0 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc590113f80 con 0x7fc5901043a0 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901039e0 0x7fc59019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc587e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 0x7fc59019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc587e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 0x7fc59019a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55540/0 (socket says 172.21.15.94:55540) 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901099f0 0x7fc59019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc587e00640 1 -- 172.21.15.94:0/2253793043 learned_addr learned my addr 172.21.15.94:0/2253793043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.593+0000 7fc58de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901039e0 0x7fc59019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34720/0 (socket says 172.21.15.94:34720) 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc587e00640 1 -- 172.21.15.94:0/2253793043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901099f0 msgr2=0x7fc59019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc587e00640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901099f0 0x7fc59019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc587e00640 1 -- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901039e0 msgr2=0x7fc59019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc587e00640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901039e0 0x7fc59019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc587e00640 1 -- 172.21.15.94:0/2253793043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59019f280 con 0x7fc5901043a0 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc58e800640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901099f0 0x7fc59019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc58de00640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901039e0 0x7fc59019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc587e00640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 0x7fc59019a820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc578006010 tx=0x7fc578003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.594+0000 7fc586a00640 1 -- 172.21.15.94:0/2253793043 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57803f070 con 0x7fc5901043a0 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.595+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59019f510 con 0x7fc5901043a0 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.595+0000 7fc586a00640 1 -- 172.21.15.94:0/2253793043 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc578031c30 con 0x7fc5901043a0 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.595+0000 7fc586a00640 1 -- 172.21.15.94:0/2253793043 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578035030 con 0x7fc5901043a0 2024-06-03T01:22:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.595+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5901a6e00 con 0x7fc5901043a0 2024-06-03T01:22:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.598+0000 7fc598005740 1 Processor -- start 2024-06-03T01:22:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.598+0000 7fc586a00640 1 -- 172.21.15.94:0/2253793043 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc578042070 con 0x7fc5901043a0 2024-06-03T01:22:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.598+0000 7fc598005740 1 -- start start 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.598+0000 7fc598005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc598005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc598005740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4d4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc598005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56097e42b420 con 0x56097e4c7200 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc598005740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56097e3d2c50 con 0x56097e4d1f80 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc598005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56097e4273e0 con 0x56097e4c8770 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55550/0 (socket says 172.21.15.94:55550) 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 -- 172.21.15.94:0/627405984 learned_addr learned my addr 172.21.15.94:0/627405984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:57.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58de00640 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc587e00640 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4d4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 -- 172.21.15.94:0/627405984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 msgr2=0x56097e4d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4d4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 -- 172.21.15.94:0/627405984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 msgr2=0x56097e4c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.599+0000 7fc58e800640 1 -- 172.21.15.94:0/627405984 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097e13f210 con 0x56097e4c8770 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.600+0000 7fc586a00640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5740a2d70 0x7fc5740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.600+0000 7fc58e800640 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c6cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc588011810 tx=0x7fc588011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6659d2d28cda5994 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.600+0000 7fc586a00640 1 -- 172.21.15.94:0/2253793043 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7fc5780efe20 con 0x7fc5901043a0 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.600+0000 7fc58de00640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5740a2d70 0x7fc5740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.600+0000 7fc57f400640 1 -- 172.21.15.94:0/627405984 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588011310 con 0x56097e4c8770 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.600+0000 7fc57f400640 1 -- 172.21.15.94:0/627405984 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588003e50 con 0x56097e4c8770 2024-06-03T01:22:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.600+0000 7fc57f400640 1 -- 172.21.15.94:0/627405984 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588004620 con 0x56097e4c8770 2024-06-03T01:22:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 -- 172.21.15.94:0/627405984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 msgr2=0x56097e4c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c6cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc588011810 tx=0x7fc588011cd0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc58de00640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5740a2d70 0x7fc5740a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x56097e4c9ed0 tx=0x7fc580005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 -- 172.21.15.94:0/627405984 shutdown_connections 2024-06-03T01:22:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4d4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 --2- 172.21.15.94:0/627405984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c6cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 -- 172.21.15.94:0/627405984 >> 172.21.15.94:0/627405984 conn(0x56097e48ff80 msgr2=0x56097e4c37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 -- 172.21.15.94:0/627405984 shutdown_connections 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 -- 172.21.15.94:0/627405984 wait complete. 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.601+0000 7fc598005740 1 -- start start 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 1 Processor -- start 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c8770 0x56097e4cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4d1f80 0x56097e4cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x56097e4d4d50 con 0x56097e4c7200 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56097e3e60b0 con 0x56097e4c8770 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56097e4d4ed0 con 0x56097e4d1f80 2024-06-03T01:22:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc598005740 10 client.0 _ll_register_callbacks cb 0x56097e3d1f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc587e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c8770 0x56097e4cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc58de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.602+0000 7fc58e800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4d1f80 0x56097e4cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc587e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c8770 0x56097e4cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:44444/0 (socket says 172.21.15.94:44444) 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc587e00640 1 -- 172.21.15.94:0/500883246 learned_addr learned my addr 172.21.15.94:0/500883246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc58e800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4d1f80 0x56097e4cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55564/0 (socket says 172.21.15.94:55564) 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc58e800640 1 -- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c8770 msgr2=0x56097e4cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc58e800640 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c8770 0x56097e4cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc58e800640 1 -- 172.21.15.94:0/500883246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 msgr2=0x56097e4cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc58e800640 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc58e800640 1 -- 172.21.15.94:0/500883246 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097e20b620 con 0x56097e4d1f80 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc587e00640 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c8770 0x56097e4cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc58e800640 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4d1f80 0x56097e4cfab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc58800a010 tx=0x7fc588011340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471d39ab02f67bf9 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc573e00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.603+0000 7fc573e00640 1 -- 172.21.15.94:0/500883246 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58800fba0 con 0x56097e4d1f80 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc573e00640 1 -- 172.21.15.94:0/500883246 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7fc58800fd40 con 0x56097e4d1f80 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc588003e50 con 0x56097e4d1f80 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc573e00640 1 -- 172.21.15.94:0/500883246 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fc588004a40 con 0x56097e4d1f80 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc598005740 10 client.26314 fetch_fsmap learned FSMap version 60 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56097e4e1160 con 0x56097e4d1f80 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc573e00640 1 -- 172.21.15.94:0/500883246 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588003e50 con 0x56097e4d1f80 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc573e00640 1 -- 172.21.15.94:0/500883246 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7fc588018020 con 0x56097e4d1f80 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc598005740 10 client.26314 fetch_fsmap finished waiting for FSMap version 60 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc598005740 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc598005740 20 client.26314 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.604+0000 7fc598005740 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e4e2510 0x56097e4e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.605+0000 7fc598005740 4 client.26314 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:22:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.605+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56097e438b50 con 0x56097e4e2510 2024-06-03T01:22:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.605+0000 7fc58de00640 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e4e2510 0x56097e4e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.605+0000 7fc58de00640 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e4e2510 0x56097e4e4900 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80691f3ee96252d3 server_cookie=80aca18988925dc7 in_seq=0 out_seq=0 2024-06-03T01:22:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.605+0000 7fc573e00640 10 client.26314 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc573e00640 1 -- 172.21.15.94:0/500883246 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56097e438b50 con 0x56097e4e2510 2024-06-03T01:22:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc573e00640 10 client.26314 handle_command_reply: tid=0 2024-06-03T01:22:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 client.26314 shutdown 2024-06-03T01:22:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4d1f80 msgr2=0x56097e4cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4d1f80 0x56097e4cfab0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc58800a010 tx=0x7fc588011340 comp rx=0 tx=0).stop 2024-06-03T01:22:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 shutdown_connections 2024-06-03T01:22:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4d1f80 0x56097e4cfab0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e4e2510 0x56097e4e4900 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c8770 0x56097e4cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 --2- 172.21.15.94:0/500883246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc573e00640 0 client.26314 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.606+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 >> 172.21.15.94:0/500883246 conn(0x56097e48ff80 msgr2=0x56097e4c9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.607+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 shutdown_connections 2024-06-03T01:22:57.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.607+0000 7fc598005740 1 -- 172.21.15.94:0/500883246 wait complete. 2024-06-03T01:22:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.607+0000 7fc598005740 20 client.26314 trim_cache size 0 max 16384 2024-06-03T01:22:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 Processor -- start 2024-06-03T01:22:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 -- start start 2024-06-03T01:22:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc5901b0e80 con 0x56097e4c7200 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5901b2a50 con 0x56097e4d1f80 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.624+0000 7fc598005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc5901b15b0 con 0x56097e4c8770 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc587e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc587e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55580/0 (socket says 172.21.15.94:55580) 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc587e00640 1 -- 172.21.15.94:0/668910823 learned_addr learned my addr 172.21.15.94:0/668910823 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58de00640 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58e800640 1 -- 172.21.15.94:0/668910823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 msgr2=0x56097e4c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58e800640 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58e800640 1 -- 172.21.15.94:0/668910823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 msgr2=0x56097e4cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58e800640 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58e800640 1 -- 172.21.15.94:0/668910823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097e4befd0 con 0x56097e4d1f80 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58de00640 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc587e00640 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.625+0000 7fc58e800640 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4e9fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc588011d10 tx=0x7fc5880058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879294ad3275d79d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.626+0000 7fc57ea00640 1 -- 172.21.15.94:0/668910823 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588005070 con 0x56097e4d1f80 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.626+0000 7fc57ea00640 1 -- 172.21.15.94:0/668910823 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588005210 con 0x56097e4d1f80 2024-06-03T01:22:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.626+0000 7fc57ea00640 1 -- 172.21.15.94:0/668910823 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588024900 con 0x56097e4d1f80 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 -- 172.21.15.94:0/668910823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 msgr2=0x56097e4e9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4e9fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc588011d10 tx=0x7fc5880058e0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 -- 172.21.15.94:0/668910823 shutdown_connections 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4d1f80 0x56097e4e9fc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 --2- 172.21.15.94:0/668910823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4c7200 0x56097e4cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 -- 172.21.15.94:0/668910823 >> 172.21.15.94:0/668910823 conn(0x56097e48ff80 msgr2=0x56097e4c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 -- 172.21.15.94:0/668910823 shutdown_connections 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 -- 172.21.15.94:0/668910823 wait complete. 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 -- start start 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.627+0000 7fc598005740 1 Processor -- start 2024-06-03T01:22:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc598005740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c7200 0x56097e4d5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc598005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc598005740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4d1f80 0x56097e544f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc598005740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc5901b2380 con 0x56097e4d1f80 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc598005740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5901b1cb0 con 0x56097e4c7200 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc598005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc5901aee80 con 0x56097e4c8770 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc598005740 10 client.0 _ll_register_callbacks cb 0x7fc580009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc587e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc58de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c7200 0x56097e4d5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc587e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4d5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:55594/0 (socket says 172.21.15.94:55594) 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc587e00640 1 -- 172.21.15.94:0/2479555986 learned_addr learned my addr 172.21.15.94:0/2479555986 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.628+0000 7fc58e800640 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4d1f80 0x56097e544f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc587e00640 1 -- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c7200 msgr2=0x56097e4d5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc587e00640 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c7200 0x56097e4d5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc587e00640 1 -- 172.21.15.94:0/2479555986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4d1f80 msgr2=0x56097e544f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc587e00640 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4d1f80 0x56097e544f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc587e00640 1 -- 172.21.15.94:0/2479555986 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097e545590 con 0x56097e4c8770 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc58de00640 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c7200 0x56097e4d5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc587e00640 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4d5e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5780b8430 tx=0x7fc5780ba4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c1f02a9df5517d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc573e00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc573e00640 1 -- 172.21.15.94:0/2479555986 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578049040 con 0x56097e4c8770 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc588005210 con 0x56097e4c8770 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc573e00640 1 -- 172.21.15.94:0/2479555986 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9475+0+0 (secure 0 0 0) 0x7fc5780ed3d0 con 0x56097e4c8770 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc573e00640 1 -- 172.21.15.94:0/2479555986 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fc5780bbb10 con 0x56097e4c8770 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc598005740 10 client.26320 fetch_fsmap learned FSMap version 60 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56097e54c920 con 0x56097e4c8770 2024-06-03T01:22:57.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.629+0000 7fc573e00640 1 -- 172.21.15.94:0/2479555986 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc588005210 con 0x56097e4c8770 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc573e00640 1 -- 172.21.15.94:0/2479555986 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 60) ==== 8386+0+0 (secure 0 0 0) 0x7fc5780bf020 con 0x56097e4c8770 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc598005740 10 client.26320 fetch_fsmap finished waiting for FSMap version 60 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc598005740 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc598005740 20 client.26320 populate_metadata read hostname 'smithi094' 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc598005740 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e54d0d0 0x56097e54f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc598005740 4 client.26320 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56097e432ea0 con 0x56097e54d0d0 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc58de00640 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e54d0d0 0x56097e54f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.630+0000 7fc58de00640 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e54d0d0 0x56097e54f4f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6687eca9318fdeb server_cookie=cbe4ca3a03a18466 in_seq=0 out_seq=0 2024-06-03T01:22:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.631+0000 7fc573e00640 10 client.26320 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc573e00640 1 -- 172.21.15.94:0/2479555986 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56097e432ea0 con 0x56097e54d0d0 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc573e00640 10 client.26320 handle_command_reply: tid=0 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 client.26320 shutdown 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 msgr2=0x56097e4d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4d5e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5780b8430 tx=0x7fc5780ba4e0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 shutdown_connections 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56097e4d1f80 0x56097e544f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56097e4c8770 0x56097e4d5e50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x56097e54d0d0 0x56097e54f4f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 --2- 172.21.15.94:0/2479555986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56097e4c7200 0x56097e4d5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc573e00640 0 client.26320 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 >> 172.21.15.94:0/2479555986 conn(0x56097e48ff80 msgr2=0x56097e4c2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 shutdown_connections 2024-06-03T01:22:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.663+0000 7fc598005740 1 -- 172.21.15.94:0/2479555986 wait complete. 2024-06-03T01:22:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.664+0000 7fc598005740 20 client.26320 trim_cache size 0 max 16384 2024-06-03T01:22:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:22:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:22:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5740a2d70 msgr2=0x7fc5740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5740a2d70 0x7fc5740a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x56097e4c9ed0 tx=0x7fc580005f90 comp rx=0 tx=0).stop 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 msgr2=0x7fc59019a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 0x7fc59019a820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc578006010 tx=0x7fc578003c10 comp rx=0 tx=0).stop 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 shutdown_connections 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc5740a2d70 0x7fc5740a5230 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5901099f0 0x7fc59019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5901043a0 0x7fc59019a820 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 --2- 172.21.15.94:0/2253793043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5901039e0 0x7fc59019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:22:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 >> 172.21.15.94:0/2253793043 conn(0x7fc5900ff4b0 msgr2=0x7fc5900ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:22:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 shutdown_connections 2024-06-03T01:22:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:22:57.665+0000 7fc58f200640 1 -- 172.21.15.94:0/2253793043 wait complete. 2024-06-03T01:22:58.143 INFO:tasks.workunit.client.0.smithi094.stdout: 1 286680 2.09 MB/sec execute 545 sec latency 1811.697 ms 2024-06-03T01:22:58.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:22:58 smithi107 ceph-mon[36622]: pgmap v393: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-06-03T01:22:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:22:58 smithi094 ceph-mon[32119]: pgmap v393: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-06-03T01:22:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:22:58 smithi179 ceph-mon[37041]: pgmap v393: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-06-03T01:22:59.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 287701 2.10 MB/sec execute 546 sec latency 21.084 ms 2024-06-03T01:23:00.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 288458 2.10 MB/sec execute 547 sec latency 18.779 ms 2024-06-03T01:23:00.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:00 smithi107 ceph-mon[36622]: pgmap v394: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-03T01:23:00.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:00 smithi094 ceph-mon[32119]: pgmap v394: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-03T01:23:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:00 smithi179 ceph-mon[37041]: pgmap v394: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-03T01:23:01.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 289024 2.11 MB/sec execute 548 sec latency 452.681 ms 2024-06-03T01:23:02.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 289024 2.10 MB/sec execute 549 sec latency 1452.823 ms 2024-06-03T01:23:02.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:02 smithi107 ceph-mon[36622]: pgmap v395: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-03T01:23:02.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:02 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:02 smithi094 ceph-mon[32119]: pgmap v395: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-03T01:23:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:02 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:02 smithi179 ceph-mon[37041]: pgmap v395: 129 pgs: 129 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-06-03T01:23:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:02 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:03.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 289605 2.10 MB/sec execute 550 sec latency 1691.727 ms 2024-06-03T01:23:04.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 289740 2.10 MB/sec execute 551 sec latency 97.782 ms 2024-06-03T01:23:04.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:04 smithi107 ceph-mon[36622]: pgmap v396: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-03T01:23:04.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:04 smithi094 ceph-mon[32119]: pgmap v396: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-03T01:23:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:04 smithi179 ceph-mon[37041]: pgmap v396: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-03T01:23:05.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 290966 2.11 MB/sec execute 552 sec latency 38.404 ms 2024-06-03T01:23:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:05 smithi094 ceph-mon[32119]: osdmap e114: 12 total, 12 up, 12 in 2024-06-03T01:23:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:05 smithi179 ceph-mon[37041]: osdmap e114: 12 total, 12 up, 12 in 2024-06-03T01:23:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:05 smithi107 ceph-mon[36622]: osdmap e114: 12 total, 12 up, 12 in 2024-06-03T01:23:06.144 INFO:tasks.workunit.client.0.smithi094.stdout: 1 291477 2.11 MB/sec execute 553 sec latency 17.726 ms 2024-06-03T01:23:06.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:06 smithi094 ceph-mon[32119]: pgmap v398: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-06-03T01:23:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:06 smithi179 ceph-mon[37041]: pgmap v398: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-06-03T01:23:06.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:06 smithi107 ceph-mon[36622]: pgmap v398: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-06-03T01:23:07.145 INFO:tasks.workunit.client.0.smithi094.stdout: 1 291477 2.11 MB/sec execute 554 sec latency 1003.667 ms 2024-06-03T01:23:08.145 INFO:tasks.workunit.client.0.smithi094.stdout: 1 292083 2.11 MB/sec execute 555 sec latency 1246.401 ms 2024-06-03T01:23:08.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:08 smithi094 ceph-mon[32119]: pgmap v399: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 858 KiB/s wr, 159 op/s 2024-06-03T01:23:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:08 smithi179 ceph-mon[37041]: pgmap v399: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 858 KiB/s wr, 159 op/s 2024-06-03T01:23:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:08 smithi107 ceph-mon[36622]: pgmap v399: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 858 KiB/s wr, 159 op/s 2024-06-03T01:23:09.145 INFO:tasks.workunit.client.0.smithi094.stdout: 1 292567 2.11 MB/sec execute 556 sec latency 518.979 ms 2024-06-03T01:23:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:09 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:09 smithi094 ceph-mon[32119]: osdmap e115: 12 total, 12 up, 12 in 2024-06-03T01:23:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:09 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:09.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:09 smithi179 ceph-mon[37041]: osdmap e115: 12 total, 12 up, 12 in 2024-06-03T01:23:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:09 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:09 smithi107 ceph-mon[36622]: osdmap e115: 12 total, 12 up, 12 in 2024-06-03T01:23:10.145 INFO:tasks.workunit.client.0.smithi094.stdout: 1 292567 2.11 MB/sec execute 557 sec latency 1519.073 ms 2024-06-03T01:23:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:10 smithi094 ceph-mon[32119]: pgmap v401: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.0 MiB/s wr, 199 op/s 2024-06-03T01:23:10.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:10 smithi179 ceph-mon[37041]: pgmap v401: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.0 MiB/s wr, 199 op/s 2024-06-03T01:23:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:10 smithi107 ceph-mon[36622]: pgmap v401: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.0 MiB/s wr, 199 op/s 2024-06-03T01:23:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 -- 172.21.15.94:0/1403447343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 msgr2=0x7f1f14101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1403447343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 0x7f1f14101e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009930 tx=0x7f1efc02a420 comp rx=0 tx=0).stop 2024-06-03T01:23:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 -- 172.21.15.94:0/1403447343 shutdown_connections 2024-06-03T01:23:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1403447343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 0x7f1f1410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1403447343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 0x7f1f14101e30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1403447343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f141079c0 0x7f1f14107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 -- 172.21.15.94:0/1403447343 >> 172.21.15.94:0/1403447343 conn(0x7f1f14078e30 msgr2=0x7f1f14079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 -- 172.21.15.94:0/1403447343 shutdown_connections 2024-06-03T01:23:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.820+0000 7f1f1b200640 1 -- 172.21.15.94:0/1403447343 wait complete. 2024-06-03T01:23:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.821+0000 7f1f1b200640 1 Processor -- start 2024-06-03T01:23:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.821+0000 7f1f1b200640 1 -- start start 2024-06-03T01:23:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f1b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 0x7f1f14073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f1b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 0x7f1f140741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f1b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f141079c0 0x7f1f14074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f1b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14111da0 con 0x7f1f141019b0 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f1b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14111c20 con 0x7f1f14102370 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f1b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14111f20 con 0x7f1f141079c0 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 0x7f1f14073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 0x7f1f140741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 0x7f1f140741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42786/0 (socket says 172.21.15.94:42786) 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13400640 1 -- 172.21.15.94:0/1078124958 learned_addr learned my addr 172.21.15.94:0/1078124958 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f19200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f141079c0 0x7f1f14074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13400640 1 -- 172.21.15.94:0/1078124958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f141079c0 msgr2=0x7f1f14074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13400640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f141079c0 0x7f1f14074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13400640 1 -- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 msgr2=0x7f1f14073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.822+0000 7f1f13400640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 0x7f1f14073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.823+0000 7f1f13400640 1 -- 172.21.15.94:0/1078124958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f141a66e0 con 0x7f1f14102370 2024-06-03T01:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.823+0000 7f1f13400640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 0x7f1f140741e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009230 tx=0x7f1efc004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.823+0000 7f1f10c00640 1 -- 172.21.15.94:0/1078124958 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc03f030 con 0x7f1f14102370 2024-06-03T01:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.823+0000 7f1f10c00640 1 -- 172.21.15.94:0/1078124958 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1efc004440 con 0x7f1f14102370 2024-06-03T01:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.823+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f141a6970 con 0x7f1f14102370 2024-06-03T01:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.823+0000 7f1f10c00640 1 -- 172.21.15.94:0/1078124958 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc035070 con 0x7f1f14102370 2024-06-03T01:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.823+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f141a6e60 con 0x7f1f14102370 2024-06-03T01:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.826+0000 7f1f10c00640 1 -- 172.21.15.94:0/1078124958 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1efc042030 con 0x7f1f14102370 2024-06-03T01:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.826+0000 7f1f1df85740 1 Processor -- start 2024-06-03T01:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f1df85740 1 -- start start 2024-06-03T01:23:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f1df85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558c8b141420 con 0x558c8b1de5b0 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f1df85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558c8b0e8c50 con 0x558c8b1dd040 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f1df85740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558c8b13d3e0 con 0x558c8b1e7dc0 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f19200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1dd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42802/0 (socket says 172.21.15.94:42802) 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.827+0000 7f1f13e00640 1 -- 172.21.15.94:0/2132487711 learned_addr learned my addr 172.21.15.94:0/2132487711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f13400640 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f13e00640 1 -- 172.21.15.94:0/2132487711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 msgr2=0x558c8b1ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f13e00640 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1ea1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f13e00640 1 -- 172.21.15.94:0/2132487711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 msgr2=0x558c8b1dcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f13e00640 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1dcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f13e00640 1 -- 172.21.15.94:0/2132487711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8ae55210 con 0x558c8b1dd040 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f13e00640 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1dd4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011440 tx=0x7f1f04011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6301e4e8998aa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f10c00640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f1ef80a2da0 0x7f1ef80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f09600640 1 -- 172.21.15.94:0/2132487711 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04003c30 con 0x558c8b1dd040 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f10c00640 1 -- 172.21.15.94:0/1078124958 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f1efc0efb70 con 0x7f1f14102370 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f09600640 1 -- 172.21.15.94:0/2132487711 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f04003dd0 con 0x558c8b1dd040 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.828+0000 7f1f09600640 1 -- 172.21.15.94:0/2132487711 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400fbf0 con 0x558c8b1dd040 2024-06-03T01:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f19200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f1ef80a2da0 0x7f1ef80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 -- 172.21.15.94:0/2132487711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 msgr2=0x558c8b1dd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1dd4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011440 tx=0x7f1f04011900 comp rx=0 tx=0).stop 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 -- 172.21.15.94:0/2132487711 shutdown_connections 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1dd4c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 --2- 172.21.15.94:0/2132487711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1dcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 -- 172.21.15.94:0/2132487711 >> 172.21.15.94:0/2132487711 conn(0x558c8b1a5f80 msgr2=0x558c8b1d9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f19200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f1ef80a2da0 0x7f1ef80a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009d90 tx=0x7f1f0c005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 -- 172.21.15.94:0/2132487711 shutdown_connections 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.829+0000 7f1f1df85740 1 -- 172.21.15.94:0/2132487711 wait complete. 2024-06-03T01:23:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.830+0000 7f1f1df85740 1 -- start start 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.830+0000 7f1f1df85740 1 Processor -- start 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1de5b0 0x558c8b1e5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f1df85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x558c8b1eab90 con 0x558c8b1e7dc0 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f1df85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558c8b0fc0b0 con 0x558c8b1dd040 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f1df85740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558c8b1ead10 con 0x558c8b1de5b0 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f1df85740 10 client.0 _ll_register_callbacks cb 0x558c8b0e7f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1e4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42814/0 (socket says 172.21.15.94:42814) 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 -- 172.21.15.94:0/3035407946 learned_addr learned my addr 172.21.15.94:0/3035407946 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f19200640 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13400640 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1de5b0 0x558c8b1e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 -- 172.21.15.94:0/3035407946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1de5b0 msgr2=0x558c8b1e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1de5b0 0x558c8b1e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 -- 172.21.15.94:0/3035407946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1e7dc0 msgr2=0x558c8b1e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.831+0000 7f1f13e00640 1 -- 172.21.15.94:0/3035407946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8af21620 con 0x558c8b1dd040 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f13e00640 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1e4e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1f040099d0 tx=0x7f1f04004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c17f0109ec5df73 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f02000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f02000640 1 -- 172.21.15.94:0/3035407946 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04026070 con 0x558c8b1dd040 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f02000640 1 -- 172.21.15.94:0/3035407946 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f1f0400fd00 con 0x558c8b1dd040 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f04003dd0 con 0x558c8b1dd040 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f02000640 1 -- 172.21.15.94:0/3035407946 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f1f04032050 con 0x558c8b1dd040 2024-06-03T01:23:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f1df85740 10 client.26318 fetch_fsmap learned FSMap version 62 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558c8b1f6f50 con 0x558c8b1dd040 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.832+0000 7f1f02000640 1 -- 172.21.15.94:0/3035407946 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f04003dd0 con 0x558c8b1dd040 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f02000640 1 -- 172.21.15.94:0/3035407946 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 62) ==== 8386+0+0 (secure 0 0 0) 0x7f1f04005040 con 0x558c8b1dd040 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f1df85740 10 client.26318 fetch_fsmap finished waiting for FSMap version 62 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f1df85740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f1df85740 20 client.26318 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b1f8300 0x558c8b1fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f1df85740 4 client.26318 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c8b14eb50 con 0x558c8b1f8300 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.833+0000 7f1f13400640 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b1f8300 0x558c8b1fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.834+0000 7f1f13400640 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b1f8300 0x558c8b1fa6f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3f5f5e42200101 server_cookie=99eaef80b67aa7b7 in_seq=0 out_seq=0 2024-06-03T01:23:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.834+0000 7f1f02000640 10 client.26318 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.835+0000 7f1f02000640 1 -- 172.21.15.94:0/3035407946 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c8b14eb50 con 0x558c8b1f8300 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.835+0000 7f1f02000640 10 client.26318 handle_command_reply: tid=0 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.835+0000 7f1f1df85740 1 client.26318 shutdown 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 msgr2=0x558c8b1e4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1e4e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1f040099d0 tx=0x7f1f04004c70 comp rx=0 tx=0).stop 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 shutdown_connections 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1de5b0 0x558c8b1e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b1f8300 0x558c8b1fa6f0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3035407946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1e4e30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f02000640 0 client.26318 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 >> 172.21.15.94:0/3035407946 conn(0x558c8b1a5f80 msgr2=0x558c8b1df4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 shutdown_connections 2024-06-03T01:23:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 1 -- 172.21.15.94:0/3035407946 wait complete. 2024-06-03T01:23:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.836+0000 7f1f1df85740 20 client.26318 trim_cache size 0 max 16384 2024-06-03T01:23:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 Processor -- start 2024-06-03T01:23:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 -- start start 2024-06-03T01:23:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b200370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141b1020 con 0x558c8b1de5b0 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141b2bf0 con 0x558c8b1dd040 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.859+0000 7f1f1df85740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141b1780 con 0x558c8b1e7dc0 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b200370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1ffe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42816/0 (socket says 172.21.15.94:42816) 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 -- 172.21.15.94:0/171790686 learned_addr learned my addr 172.21.15.94:0/171790686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f19200640 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 -- 172.21.15.94:0/171790686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 msgr2=0x558c8b1e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 -- 172.21.15.94:0/171790686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 msgr2=0x558c8b200370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b200370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 -- 172.21.15.94:0/171790686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8b1e3380 con 0x558c8b1dd040 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13400640 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b200370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.860+0000 7f1f13e00640 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1ffe30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009d60 tx=0x7f1f04005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e084a35ec971bd server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.861+0000 7f1f08c00640 1 -- 172.21.15.94:0/171790686 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04026040 con 0x558c8b1dd040 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.861+0000 7f1f08c00640 1 -- 172.21.15.94:0/171790686 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f040094e0 con 0x558c8b1dd040 2024-06-03T01:23:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.861+0000 7f1f08c00640 1 -- 172.21.15.94:0/171790686 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04004000 con 0x558c8b1dd040 2024-06-03T01:23:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 -- 172.21.15.94:0/171790686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 msgr2=0x558c8b1ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1ffe30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009d60 tx=0x7f1f04005f30 comp rx=0 tx=0).stop 2024-06-03T01:23:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 -- 172.21.15.94:0/171790686 shutdown_connections 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1e7dc0 0x558c8b1e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b200370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 --2- 172.21.15.94:0/171790686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1dd040 0x558c8b1ffe30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 -- 172.21.15.94:0/171790686 >> 172.21.15.94:0/171790686 conn(0x558c8b1a5f80 msgr2=0x558c8b1e3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 -- 172.21.15.94:0/171790686 shutdown_connections 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 -- 172.21.15.94:0/171790686 wait complete. 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 -- start start 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.862+0000 7f1f1df85740 1 Processor -- start 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1dd040 0x558c8b1eb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f1df85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1e7dc0 0x558c8b260df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f1df85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141b2520 con 0x558c8b1de5b0 2024-06-03T01:23:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f1df85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141b1e50 con 0x558c8b1e7dc0 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f1df85740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141af020 con 0x558c8b1dd040 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f1df85740 10 client.0 _ll_register_callbacks cb 0x7f1f0c009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f19200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1e7dc0 0x558c8b260df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f19200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1e7dc0 0x558c8b260df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:42830/0 (socket says 172.21.15.94:42830) 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f19200640 1 -- 172.21.15.94:0/3309738845 learned_addr learned my addr 172.21.15.94:0/3309738845 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f13400640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.863+0000 7f1f13e00640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1dd040 0x558c8b1eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f19200640 1 -- 172.21.15.94:0/3309738845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1dd040 msgr2=0x558c8b1eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f19200640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1dd040 0x558c8b1eb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f19200640 1 -- 172.21.15.94:0/3309738845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 msgr2=0x558c8b1ebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f19200640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1ebaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f19200640 1 -- 172.21.15.94:0/3309738845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8b261380 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f13400640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f19200640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1e7dc0 0x558c8b260df0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c004620 tx=0x7f1f0c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a13960aaf716cfe server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f02000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f02000640 1 -- 172.21.15.94:0/3309738845 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c003e50 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f040094e0 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.864+0000 7f1f02000640 1 -- 172.21.15.94:0/3309738845 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f1f0c005020 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f02000640 1 -- 172.21.15.94:0/3309738845 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f1f0c036050 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 10 client.26324 fetch_fsmap learned FSMap version 62 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x558c8b262740 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f02000640 1 -- 172.21.15.94:0/3309738845 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f040094e0 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f02000640 1 -- 172.21.15.94:0/3309738845 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 62) ==== 8386+0+0 (secure 0 0 0) 0x7f1f0c02b460 con 0x558c8b1e7dc0 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 10 client.26324 fetch_fsmap finished waiting for FSMap version 62 2024-06-03T01:23:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 10 client.26324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 20 client.26324 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b262f20 0x558c8b265340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 4 client.26324 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:23:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.865+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c8b148ea0 con 0x558c8b262f20 2024-06-03T01:23:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.866+0000 7f1f13e00640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b262f20 0x558c8b265340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.866+0000 7f1f02000640 10 client.26324 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.866+0000 7f1f13e00640 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b262f20 0x558c8b265340 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b272fdb66ea656 server_cookie=f6e2709a3d176b8a in_seq=0 out_seq=0 2024-06-03T01:23:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.908+0000 7f1f02000640 1 -- 172.21.15.94:0/3309738845 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558c8b148ea0 con 0x558c8b262f20 2024-06-03T01:23:10.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.908+0000 7f1f02000640 10 client.26324 handle_command_reply: tid=0 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.908+0000 7f1f1df85740 1 client.26324 shutdown 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1e7dc0 msgr2=0x558c8b260df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1e7dc0 0x558c8b260df0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c004620 tx=0x7f1f0c004730 comp rx=0 tx=0).stop 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 shutdown_connections 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x558c8b262f20 0x558c8b265340 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558c8b1e7dc0 0x558c8b260df0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f02000640 0 client.26324 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558c8b1de5b0 0x558c8b1ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 --2- 172.21.15.94:0/3309738845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c8b1dd040 0x558c8b1eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 >> 172.21.15.94:0/3309738845 conn(0x558c8b1a5f80 msgr2=0x558c8b1e4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 shutdown_connections 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 1 -- 172.21.15.94:0/3309738845 wait complete. 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.909+0000 7f1f1df85740 20 client.26324 trim_cache size 0 max 16384 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:23:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:23:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.910+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f1ef80a2da0 msgr2=0x7f1ef80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.910+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f1ef80a2da0 0x7f1ef80a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009d90 tx=0x7f1f0c005f90 comp rx=0 tx=0).stop 2024-06-03T01:23:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.910+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 msgr2=0x7f1f140741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.910+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 0x7f1f140741e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009230 tx=0x7f1efc004040 comp rx=0 tx=0).stop 2024-06-03T01:23:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 shutdown_connections 2024-06-03T01:23:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f1ef80a2da0 0x7f1ef80a5260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f141079c0 0x7f1f14074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f14102370 0x7f1f140741e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 --2- 172.21.15.94:0/1078124958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f141019b0 0x7f1f14073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 >> 172.21.15.94:0/1078124958 conn(0x7f1f14078e30 msgr2=0x7f1f140ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 shutdown_connections 2024-06-03T01:23:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:10.911+0000 7f1f1b200640 1 -- 172.21.15.94:0/1078124958 wait complete. 2024-06-03T01:23:11.145 INFO:tasks.workunit.client.0.smithi094.stdout: 1 293753 2.11 MB/sec execute 558 sec latency 1706.425 ms 2024-06-03T01:23:12.145 INFO:tasks.workunit.client.0.smithi094.stdout: 1 294731 2.12 MB/sec execute 559 sec latency 18.505 ms 2024-06-03T01:23:12.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:12 smithi094 ceph-mon[32119]: pgmap v402: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 535 KiB/s wr, 105 op/s 2024-06-03T01:23:12.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:12 smithi179 ceph-mon[37041]: pgmap v402: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 535 KiB/s wr, 105 op/s 2024-06-03T01:23:12.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:12 smithi107 ceph-mon[36622]: pgmap v402: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 535 KiB/s wr, 105 op/s 2024-06-03T01:23:13.145 INFO:tasks.workunit.client.0.smithi094.stdout: 1 295338 2.12 MB/sec execute 560 sec latency 17.770 ms 2024-06-03T01:23:14.146 INFO:tasks.workunit.client.0.smithi094.stdout: 1 296117 2.13 MB/sec execute 561 sec latency 288.712 ms 2024-06-03T01:23:14.604 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:14 smithi094 ceph-mon[32119]: pgmap v403: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-06-03T01:23:14.605 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:14 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:14.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:14 smithi179 ceph-mon[37041]: pgmap v403: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-06-03T01:23:14.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:14 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:14 smithi107 ceph-mon[36622]: pgmap v403: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-06-03T01:23:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:14 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:15.146 INFO:tasks.workunit.client.0.smithi094.stdout: 1 296117 2.12 MB/sec execute 562 sec latency 1288.879 ms 2024-06-03T01:23:16.146 INFO:tasks.workunit.client.0.smithi094.stdout: 1 297309 2.13 MB/sec execute 563 sec latency 1464.889 ms 2024-06-03T01:23:16.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:16 smithi094 ceph-mon[32119]: pgmap v404: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-06-03T01:23:16.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:16 smithi179 ceph-mon[37041]: pgmap v404: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-06-03T01:23:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:16 smithi107 ceph-mon[36622]: pgmap v404: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-06-03T01:23:17.146 INFO:tasks.workunit.client.0.smithi094.stdout: 1 298277 2.14 MB/sec execute 564 sec latency 50.110 ms 2024-06-03T01:23:18.146 INFO:tasks.workunit.client.0.smithi094.stdout: 1 298888 2.14 MB/sec execute 565 sec latency 20.983 ms 2024-06-03T01:23:18.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:18 smithi094 ceph-mon[32119]: pgmap v405: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-03T01:23:18.604 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:18 smithi179 ceph-mon[37041]: pgmap v405: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-03T01:23:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:18 smithi107 ceph-mon[36622]: pgmap v405: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-03T01:23:19.146 INFO:tasks.workunit.client.0.smithi094.stdout: 1 299665 2.14 MB/sec execute 566 sec latency 257.995 ms 2024-06-03T01:23:20.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 299665 2.14 MB/sec execute 567 sec latency 1258.138 ms 2024-06-03T01:23:20.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:20 smithi094 ceph-mon[32119]: pgmap v406: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-06-03T01:23:20.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:20 smithi179 ceph-mon[37041]: pgmap v406: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-06-03T01:23:20.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:20 smithi107 ceph-mon[36622]: pgmap v406: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-06-03T01:23:21.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 299799 2.13 MB/sec execute 568 sec latency 2174.130 ms 2024-06-03T01:23:22.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 300984 2.14 MB/sec execute 569 sec latency 41.003 ms 2024-06-03T01:23:22.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:22 smithi179 ceph-mon[37041]: pgmap v407: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-06-03T01:23:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:22 smithi107 ceph-mon[36622]: pgmap v407: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-06-03T01:23:22.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:22 smithi094 ceph-mon[32119]: pgmap v407: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-06-03T01:23:23.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 301911 2.15 MB/sec execute 570 sec latency 22.487 ms 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.062+0000 7f23be000640 1 -- 172.21.15.94:0/2191489252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b8102370 msgr2=0x7f23b810e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.062+0000 7f23be000640 1 --2- 172.21.15.94:0/2191489252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b8102370 0x7f23b810e880 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009cd0 tx=0x7f23b002a8f0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.062+0000 7f23be000640 1 -- 172.21.15.94:0/2191489252 shutdown_connections 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.062+0000 7f23be000640 1 --2- 172.21.15.94:0/2191489252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b8102370 0x7f23b810e880 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.062+0000 7f23be000640 1 --2- 172.21.15.94:0/2191489252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b81019b0 0x7f23b8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.062+0000 7f23be000640 1 --2- 172.21.15.94:0/2191489252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b81079c0 0x7f23b8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.062+0000 7f23be000640 1 -- 172.21.15.94:0/2191489252 >> 172.21.15.94:0/2191489252 conn(0x7f23b8078e30 msgr2=0x7f23b8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.063+0000 7f23be000640 1 -- 172.21.15.94:0/2191489252 shutdown_connections 2024-06-03T01:23:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.063+0000 7f23be000640 1 -- 172.21.15.94:0/2191489252 wait complete. 2024-06-03T01:23:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 Processor -- start 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 -- start start 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b81019b0 0x7f23b819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b8102370 0x7f23b819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b81079c0 0x7f23b819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8075140 con 0x7f23b81079c0 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8074fc0 con 0x7f23b8102370 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23be000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23b80752c0 con 0x7f23b81019b0 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b8102370 0x7f23b819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23b7400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b81019b0 0x7f23b819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b8102370 0x7f23b819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51358/0 (socket says 172.21.15.94:51358) 2024-06-03T01:23:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23b6a00640 1 -- 172.21.15.94:0/70338300 learned_addr learned my addr 172.21.15.94:0/70338300 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.064+0000 7f23b7e00640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b81079c0 0x7f23b819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23b7400640 1 -- 172.21.15.94:0/70338300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b8102370 msgr2=0x7f23b819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23b7400640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b8102370 0x7f23b819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23b7400640 1 -- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b81079c0 msgr2=0x7f23b819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23b7400640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b81079c0 0x7f23b819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23b7400640 1 -- 172.21.15.94:0/70338300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b819f380 con 0x7f23b81019b0 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23b6a00640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b8102370 0x7f23b819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23b7400640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b81019b0 0x7f23b819a390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f23a800e9f0 tx=0x7f23a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23afe00640 1 -- 172.21.15.94:0/70338300 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a800ccc0 con 0x7f23b81019b0 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23afe00640 1 -- 172.21.15.94:0/70338300 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a8005020 con 0x7f23b81019b0 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23afe00640 1 -- 172.21.15.94:0/70338300 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80049e0 con 0x7f23b81019b0 2024-06-03T01:23:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.065+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b819f670 con 0x7f23b81019b0 2024-06-03T01:23:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.066+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b81a6f00 con 0x7f23b81019b0 2024-06-03T01:23:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.069+0000 7f23afe00640 1 -- 172.21.15.94:0/70338300 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f23a8005560 con 0x7f23b81019b0 2024-06-03T01:23:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.069+0000 7f23c0e2f740 1 Processor -- start 2024-06-03T01:23:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.069+0000 7f23c0e2f740 1 -- start start 2024-06-03T01:23:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d07fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d0804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a03cfe4420 con 0x55a03d0815b0 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a03cf8bc50 con 0x55a03d08adc0 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a03cfe03e0 con 0x55a03d080040 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d07fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51364/0 (socket says 172.21.15.94:51364) 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23b6a00640 1 -- 172.21.15.94:0/86855403 learned_addr learned my addr 172.21.15.94:0/86855403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.070+0000 7f23b7400640 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d0804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23b6a00640 1 -- 172.21.15.94:0/86855403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 msgr2=0x55a03d0804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23b6a00640 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d0804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23b6a00640 1 -- 172.21.15.94:0/86855403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 msgr2=0x55a03d07fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23b6a00640 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d07fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23b6a00640 1 -- 172.21.15.94:0/86855403 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03ccf8210 con 0x55a03d08adc0 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23afe00640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f239c0a2da0 0x7f239c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23b6a00640 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08d1b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f23a4011620 tx=0x7f23a4011ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee5bb1e9d2a6c50 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23afe00640 1 -- 172.21.15.94:0/70338300 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f23a80cf730 con 0x7f23b81019b0 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23a3e00640 1 -- 172.21.15.94:0/86855403 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a401e070 con 0x55a03d08adc0 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23a3e00640 1 -- 172.21.15.94:0/86855403 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a400fbd0 con 0x55a03d08adc0 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23b7e00640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f239c0a2da0 0x7f239c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.071+0000 7f23a3e00640 1 -- 172.21.15.94:0/86855403 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4019d50 con 0x55a03d08adc0 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23c0e2f740 1 -- 172.21.15.94:0/86855403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 msgr2=0x55a03d08d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08d1b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f23a4011620 tx=0x7f23a4011ae0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23b7e00640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f239c0a2da0 0x7f239c0a5260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23b0004af0 tx=0x7f23b0005d80 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23c0e2f740 1 -- 172.21.15.94:0/86855403 shutdown_connections 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08d1b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d0804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/86855403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d07fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.072+0000 7f23c0e2f740 1 -- 172.21.15.94:0/86855403 >> 172.21.15.94:0/86855403 conn(0x55a03d048f80 msgr2=0x55a03d07c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.073+0000 7f23c0e2f740 1 -- 172.21.15.94:0/86855403 shutdown_connections 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.073+0000 7f23c0e2f740 1 -- 172.21.15.94:0/86855403 wait complete. 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.073+0000 7f23c0e2f740 1 -- start start 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.073+0000 7f23c0e2f740 1 Processor -- start 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d087e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d0815b0 0x55a03d0883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d08adc0 0x55a03d0888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a03d08db90 con 0x55a03d080040 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a03cf9f0b0 con 0x55a03d0815b0 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a03d08dd10 con 0x55a03d08adc0 2024-06-03T01:23:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23c0e2f740 10 client.0 _ll_register_callbacks cb 0x55a03cf8af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d08adc0 0x55a03d0888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23b7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d087e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d0815b0 0x55a03d0883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d08adc0 0x55a03d0888f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:37378/0 (socket says 172.21.15.94:37378) 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.074+0000 7f23b7e00640 1 -- 172.21.15.94:0/2651202830 learned_addr learned my addr 172.21.15.94:0/2651202830 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23b6a00640 1 -- 172.21.15.94:0/2651202830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d08adc0 msgr2=0x55a03d0888f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23b6a00640 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d08adc0 0x55a03d0888f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23b6a00640 1 -- 172.21.15.94:0/2651202830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 msgr2=0x55a03d087e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23b6a00640 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d087e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23b6a00640 1 -- 172.21.15.94:0/2651202830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03cdc4620 con 0x55a03d0815b0 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23b6a00640 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d0815b0 0x55a03d0883b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f23a400f3c0 tx=0x7f23a4005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1519d1ae4ffb26a server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23a0c00640 1 -- 172.21.15.94:0/2651202830 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a401e040 con 0x55a03d0815b0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23a0c00640 1 -- 172.21.15.94:0/2651202830 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f23a400fd80 con 0x55a03d0815b0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a400fbd0 con 0x55a03d0815b0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.075+0000 7f23a0c00640 1 -- 172.21.15.94:0/2651202830 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f23a4019a50 con 0x55a03d0815b0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 10 client.26336 fetch_fsmap learned FSMap version 64 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a03d099fa0 con 0x55a03d0815b0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23a0c00640 1 -- 172.21.15.94:0/2651202830 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a400fbd0 con 0x55a03d0815b0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23a0c00640 1 -- 172.21.15.94:0/2651202830 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f23a4021070 con 0x55a03d0815b0 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 10 client.26336 fetch_fsmap finished waiting for FSMap version 64 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 20 client.26336 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d09b350 0x55a03d09d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 4 client.26336 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:23:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a03cff1b50 con 0x55a03d09b350 2024-06-03T01:23:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.076+0000 7f23b7400640 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d09b350 0x55a03d09d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.077+0000 7f23b7400640 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d09b350 0x55a03d09d740 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc93c564c0f2777 server_cookie=cfa7c1b0de9ef5a8 in_seq=0 out_seq=0 2024-06-03T01:23:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.077+0000 7f23a0c00640 10 client.26336 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23a0c00640 1 -- 172.21.15.94:0/2651202830 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a03cff1b50 con 0x55a03d09b350 2024-06-03T01:23:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23a0c00640 10 client.26336 handle_command_reply: tid=0 2024-06-03T01:23:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 client.26336 shutdown 2024-06-03T01:23:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d0815b0 msgr2=0x55a03d0883b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d0815b0 0x55a03d0883b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f23a400f3c0 tx=0x7f23a4005c50 comp rx=0 tx=0).stop 2024-06-03T01:23:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 shutdown_connections 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d08adc0 0x55a03d0888f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d09b350 0x55a03d09d740 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d0815b0 0x55a03d0883b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2651202830 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d087e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23a0c00640 0 client.26336 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 >> 172.21.15.94:0/2651202830 conn(0x55a03d048f80 msgr2=0x55a03d0824a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 shutdown_connections 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.078+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2651202830 wait complete. 2024-06-03T01:23:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.079+0000 7f23c0e2f740 20 client.26336 trim_cache size 0 max 16384 2024-06-03T01:23:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 Processor -- start 2024-06-03T01:23:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 -- start start 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d07bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d0815b0 0x55a03d07c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d0a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81b1040 con 0x55a03d080040 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81b2c40 con 0x55a03d08adc0 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81b17d0 con 0x55a03d0815b0 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.097+0000 7f23b6a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d0815b0 0x55a03d07c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d0815b0 0x55a03d07c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:37386/0 (socket says 172.21.15.94:37386) 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 -- 172.21.15.94:0/1058474172 learned_addr learned my addr 172.21.15.94:0/1058474172 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b7400640 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d07bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b7e00640 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d0a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 -- 172.21.15.94:0/1058474172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 msgr2=0x55a03d0a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d0a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 -- 172.21.15.94:0/1058474172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 msgr2=0x55a03d07bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d07bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 -- 172.21.15.94:0/1058474172 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a03d077ff0 con 0x55a03d0815b0 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23b6a00640 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d0815b0 0x55a03d07c4d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23a4019640 tx=0x7f23a4009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a5ec4edde42cc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23a3400640 1 -- 172.21.15.94:0/1058474172 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4015070 con 0x55a03d0815b0 2024-06-03T01:23:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23a3400640 1 -- 172.21.15.94:0/1058474172 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a4005450 con 0x55a03d0815b0 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.098+0000 7f23a3400640 1 -- 172.21.15.94:0/1058474172 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a401e040 con 0x55a03d0815b0 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.099+0000 7f23c0e2f740 1 -- 172.21.15.94:0/1058474172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d0815b0 msgr2=0x55a03d07c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.099+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d0815b0 0x55a03d07c4d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23a4019640 tx=0x7f23a4009550 comp rx=0 tx=0).stop 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.099+0000 7f23c0e2f740 1 -- 172.21.15.94:0/1058474172 shutdown_connections 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.099+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d0a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.099+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d0815b0 0x55a03d07c4d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.099+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/1058474172 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d080040 0x55a03d07bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.099+0000 7f23c0e2f740 1 -- 172.21.15.94:0/1058474172 >> 172.21.15.94:0/1058474172 conn(0x55a03d048f80 msgr2=0x55a03d0886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.100+0000 7f23c0e2f740 1 -- 172.21.15.94:0/1058474172 shutdown_connections 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.100+0000 7f23c0e2f740 1 -- 172.21.15.94:0/1058474172 wait complete. 2024-06-03T01:23:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.100+0000 7f23c0e2f740 1 -- start start 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.100+0000 7f23c0e2f740 1 Processor -- start 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d08ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d08f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23c0e2f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81b2570 con 0x55a03d0815b0 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81b1ea0 con 0x55a03d08adc0 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23c0e2f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23b81af040 con 0x55a03d080040 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23c0e2f740 10 client.0 _ll_register_callbacks cb 0x7f23a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23b7400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d08ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23b7400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d08ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:37400/0 (socket says 172.21.15.94:37400) 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23b7400640 1 -- 172.21.15.94:0/2557022758 learned_addr learned my addr 172.21.15.94:0/2557022758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.101+0000 7f23b6a00640 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d08f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23b7400640 1 -- 172.21.15.94:0/2557022758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 msgr2=0x55a03d08f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23b7400640 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23b7400640 1 -- 172.21.15.94:0/2557022758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 msgr2=0x55a03d08f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23b7400640 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d08f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23b7400640 1 -- 172.21.15.94:0/2557022758 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03d104da0 con 0x55a03d080040 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23b6a00640 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d08f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23b7400640 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d08ed10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23a809b7c0 tx=0x7f23a8086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f367292825e5295e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:23:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23a0c00640 1 -- 172.21.15.94:0/2557022758 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a801c040 con 0x55a03d080040 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23a0c00640 1 -- 172.21.15.94:0/2557022758 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f23a8097ea0 con 0x55a03d080040 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a4005450 con 0x55a03d080040 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23a0c00640 1 -- 172.21.15.94:0/2557022758 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f23a809b830 con 0x55a03d080040 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23c0e2f740 10 client.26356 fetch_fsmap learned FSMap version 64 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a03d1060b0 con 0x55a03d080040 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.102+0000 7f23a0c00640 1 -- 172.21.15.94:0/2557022758 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a4005450 con 0x55a03d080040 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23a0c00640 1 -- 172.21.15.94:0/2557022758 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f23a8004be0 con 0x55a03d080040 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23c0e2f740 10 client.26356 fetch_fsmap finished waiting for FSMap version 64 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23c0e2f740 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23c0e2f740 20 client.26356 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d1067b0 0x55a03d108bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23c0e2f740 4 client.26356 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:23:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a03cfebea0 con 0x55a03d1067b0 2024-06-03T01:23:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.103+0000 7f23b6a00640 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d1067b0 0x55a03d108bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.104+0000 7f23b6a00640 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d1067b0 0x55a03d108bd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a3e91183a71165 server_cookie=d15b46c9b99ef9fb in_seq=0 out_seq=0 2024-06-03T01:23:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.104+0000 7f23a0c00640 10 client.26356 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23a0c00640 1 -- 172.21.15.94:0/2557022758 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a03cfebea0 con 0x55a03d1067b0 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23a0c00640 10 client.26356 handle_command_reply: tid=0 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 client.26356 shutdown 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 msgr2=0x55a03d08ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d08ed10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23a809b7c0 tx=0x7f23a8086040 comp rx=0 tx=0).stop 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 shutdown_connections 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x55a03d1067b0 0x55a03d108bd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a03d08adc0 0x55a03d08f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a03d0815b0 0x55a03d08f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23a0c00640 0 client.26356 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 --2- 172.21.15.94:0/2557022758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a03d080040 0x55a03d08ed10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 >> 172.21.15.94:0/2557022758 conn(0x55a03d048f80 msgr2=0x55a03d081c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 shutdown_connections 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.143+0000 7f23c0e2f740 1 -- 172.21.15.94:0/2557022758 wait complete. 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.144+0000 7f23c0e2f740 20 client.26356 trim_cache size 0 max 16384 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:23:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:23:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.144+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f239c0a2da0 msgr2=0x7f239c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.144+0000 7f23be000640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f239c0a2da0 0x7f239c0a5260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23b0004af0 tx=0x7f23b0005d80 comp rx=0 tx=0).stop 2024-06-03T01:23:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.144+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b81019b0 msgr2=0x7f23b819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.144+0000 7f23be000640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b81019b0 0x7f23b819a390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f23a800e9f0 tx=0x7f23a800eeb0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 shutdown_connections 2024-06-03T01:23:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f239c0a2da0 0x7f239c0a5260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b81079c0 0x7f23b819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23b8102370 0x7f23b819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 --2- 172.21.15.94:0/70338300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b81019b0 0x7f23b819a390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 >> 172.21.15.94:0/70338300 conn(0x7f23b8078e30 msgr2=0x7f23b80feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 shutdown_connections 2024-06-03T01:23:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:24.145+0000 7f23be000640 1 -- 172.21.15.94:0/70338300 wait complete. 2024-06-03T01:23:24.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 302591 2.15 MB/sec execute 571 sec latency 32.071 ms 2024-06-03T01:23:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:24 smithi107 ceph-mon[36622]: pgmap v408: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-06-03T01:23:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:24 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:24.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:24 smithi094 ceph-mon[32119]: pgmap v408: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-06-03T01:23:24.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:24 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:24 smithi179 ceph-mon[37041]: pgmap v408: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-06-03T01:23:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:24 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:25.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 303214 2.15 MB/sec execute 572 sec latency 452.809 ms 2024-06-03T01:23:26.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 304288 2.16 MB/sec execute 573 sec latency 631.988 ms 2024-06-03T01:23:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:26 smithi107 ceph-mon[36622]: pgmap v409: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-03T01:23:26.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:26 smithi094 ceph-mon[32119]: pgmap v409: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-03T01:23:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:26 smithi179 ceph-mon[37041]: pgmap v409: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-03T01:23:27.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 305313 2.17 MB/sec execute 574 sec latency 63.312 ms 2024-06-03T01:23:28.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 305846 2.17 MB/sec execute 575 sec latency 18.482 ms 2024-06-03T01:23:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:28 smithi107 ceph-mon[36622]: pgmap v410: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-03T01:23:28.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:28 smithi094 ceph-mon[32119]: pgmap v410: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-03T01:23:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:28 smithi179 ceph-mon[37041]: pgmap v410: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-03T01:23:29.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 306757 2.17 MB/sec execute 576 sec latency 49.564 ms 2024-06-03T01:23:30.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:29 smithi107 ceph-mon[36622]: pgmap v411: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-06-03T01:23:30.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:29 smithi094 ceph-mon[32119]: pgmap v411: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-06-03T01:23:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:29 smithi179 ceph-mon[37041]: pgmap v411: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-06-03T01:23:30.147 INFO:tasks.workunit.client.0.smithi094.stdout: 1 306757 2.17 MB/sec execute 577 sec latency 1049.706 ms 2024-06-03T01:23:31.148 INFO:tasks.workunit.client.0.smithi094.stdout: 1 306841 2.16 MB/sec execute 578 sec latency 1976.569 ms 2024-06-03T01:23:32.148 INFO:tasks.workunit.client.0.smithi094.stdout: 1 308076 2.17 MB/sec execute 579 sec latency 50.869 ms 2024-06-03T01:23:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:32 smithi107 ceph-mon[36622]: pgmap v412: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-06-03T01:23:32.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:32 smithi094 ceph-mon[32119]: pgmap v412: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-06-03T01:23:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:32 smithi179 ceph-mon[37041]: pgmap v412: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 945 KiB/s wr, 124 op/s 2024-06-03T01:23:33.148 INFO:tasks.workunit.client.0.smithi094.stdout: 1 308980 2.18 MB/sec execute 580 sec latency 34.771 ms 2024-06-03T01:23:34.148 INFO:tasks.workunit.client.0.smithi094.stdout: 1 309591 2.18 MB/sec execute 581 sec latency 22.545 ms 2024-06-03T01:23:34.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:34 smithi107 ceph-mon[36622]: pgmap v413: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-06-03T01:23:34.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:34 smithi094 ceph-mon[32119]: pgmap v413: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-06-03T01:23:34.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:34 smithi179 ceph-mon[37041]: pgmap v413: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-06-03T01:23:35.148 INFO:tasks.workunit.client.0.smithi094.stdout: 1 310305 2.18 MB/sec execute 582 sec latency 274.354 ms 2024-06-03T01:23:36.148 INFO:tasks.workunit.client.0.smithi094.stdout: 1 311491 2.19 MB/sec execute 583 sec latency 457.795 ms 2024-06-03T01:23:36.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:36 smithi107 ceph-mon[36622]: pgmap v414: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-03T01:23:36.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:36 smithi094 ceph-mon[32119]: pgmap v414: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-03T01:23:36.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:36 smithi179 ceph-mon[37041]: pgmap v414: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-03T01:23:37.149 INFO:tasks.workunit.client.0.smithi094.stdout: 1 312435 2.20 MB/sec execute 584 sec latency 17.268 ms 2024-06-03T01:23:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 -- 172.21.15.94:0/803557367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 msgr2=0x7f94dc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 --2- 172.21.15.94:0/803557367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 0x7f94dc107b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009c10 tx=0x7f94cc02a6e0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 -- 172.21.15.94:0/803557367 shutdown_connections 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 --2- 172.21.15.94:0/803557367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc102160 0x7f94dc10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 --2- 172.21.15.94:0/803557367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc1017a0 0x7f94dc101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 --2- 172.21.15.94:0/803557367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 0x7f94dc107b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 -- 172.21.15.94:0/803557367 >> 172.21.15.94:0/803557367 conn(0x7f94dc0fd2f0 msgr2=0x7f94dc0ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 -- 172.21.15.94:0/803557367 shutdown_connections 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.291+0000 7f94e2200640 1 -- 172.21.15.94:0/803557367 wait complete. 2024-06-03T01:23:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.293+0000 7f94e2200640 1 Processor -- start 2024-06-03T01:23:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.293+0000 7f94e2200640 1 -- start start 2024-06-03T01:23:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94e2200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1017a0 0x7f94dc1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94e2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc102160 0x7f94dc1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94e2200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 0x7f94dc10d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94e2200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc111b80 con 0x7f94dc1077b0 2024-06-03T01:23:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94e2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc111a00 con 0x7f94dc102160 2024-06-03T01:23:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94e2200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc111d00 con 0x7f94dc1017a0 2024-06-03T01:23:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc102160 0x7f94dc1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94e0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1017a0 0x7f94dc1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc102160 0x7f94dc1a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51908/0 (socket says 172.21.15.94:51908) 2024-06-03T01:23:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.294+0000 7f94d3e00640 1 -- 172.21.15.94:0/1418618321 learned_addr learned my addr 172.21.15.94:0/1418618321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.295+0000 7f94e1800640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 0x7f94dc10d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.295+0000 7f94e0e00640 1 -- 172.21.15.94:0/1418618321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc102160 msgr2=0x7f94dc1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.295+0000 7f94e0e00640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc102160 0x7f94dc1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.295+0000 7f94e0e00640 1 -- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 msgr2=0x7f94dc10d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.295+0000 7f94e0e00640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 0x7f94dc10d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.295+0000 7f94e0e00640 1 -- 172.21.15.94:0/1418618321 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc10e1a0 con 0x7f94dc1017a0 2024-06-03T01:23:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.296+0000 7f94d3e00640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc102160 0x7f94dc1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.296+0000 7f94e0e00640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1017a0 0x7f94dc1a4400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009b30 tx=0x7f94cc0305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.296+0000 7f94da000640 1 -- 172.21.15.94:0/1418618321 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc041020 con 0x7f94dc1017a0 2024-06-03T01:23:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.296+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc10e430 con 0x7f94dc1017a0 2024-06-03T01:23:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.297+0000 7f94da000640 1 -- 172.21.15.94:0/1418618321 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc003e50 con 0x7f94dc1017a0 2024-06-03T01:23:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.297+0000 7f94da000640 1 -- 172.21.15.94:0/1418618321 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc036040 con 0x7f94dc1017a0 2024-06-03T01:23:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.297+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1ab3c0 con 0x7f94dc1017a0 2024-06-03T01:23:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.302+0000 7f94da000640 1 -- 172.21.15.94:0/1418618321 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f94cc044020 con 0x7f94dc1017a0 2024-06-03T01:23:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.302+0000 7f94e6603740 1 Processor -- start 2024-06-03T01:23:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e6603740 1 -- start start 2024-06-03T01:23:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e6603740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b828cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e6603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b829680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e6603740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b836370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e6603740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a0b78d420 con 0x562a0b833f80 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e6603740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a0b734c50 con 0x562a0b82a770 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e6603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a0b7893e0 con 0x562a0b829200 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94da000640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f94c00a2d70 0x7f94c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b828cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.303+0000 7f94e0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b829680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b828cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:51912/0 (socket says 172.21.15.94:51912) 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e1800640 1 -- 172.21.15.94:0/2892771189 learned_addr learned my addr 172.21.15.94:0/2892771189 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94da000640 1 -- 172.21.15.94:0/1418618321 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f94cc0f03a0 con 0x7f94dc1017a0 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94d3e00640 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b836370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e1800640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f94c00a2d70 0x7f94c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e0e00640 1 -- 172.21.15.94:0/2892771189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 msgr2=0x562a0b828cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e0e00640 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b828cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e0e00640 1 -- 172.21.15.94:0/2892771189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 msgr2=0x562a0b836370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e0e00640 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b836370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e0e00640 1 -- 172.21.15.94:0/2892771189 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0b4a1210 con 0x562a0b829200 2024-06-03T01:23:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e0e00640 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b829680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ed700 tx=0x7f94cc04bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f26ff809605504f server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94e1800640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f94c00a2d70 0x7f94c00a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94d40097d0 tx=0x7f94d4005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.304+0000 7f94d2000640 1 -- 172.21.15.94:0/2892771189 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0bcc00 con 0x562a0b829200 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94d2000640 1 -- 172.21.15.94:0/2892771189 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc0bc500 con 0x562a0b829200 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94d2000640 1 -- 172.21.15.94:0/2892771189 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0b8b70 con 0x562a0b829200 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94e6603740 1 -- 172.21.15.94:0/2892771189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 msgr2=0x562a0b829680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94e6603740 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b829680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ed700 tx=0x7f94cc04bdd0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94e6603740 1 -- 172.21.15.94:0/2892771189 shutdown_connections 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94e6603740 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b836370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94e6603740 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b829680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94e6603740 1 --2- 172.21.15.94:0/2892771189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b828cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.305+0000 7f94e6603740 1 -- 172.21.15.94:0/2892771189 >> 172.21.15.94:0/2892771189 conn(0x562a0b7f1f80 msgr2=0x562a0b8257f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.306+0000 7f94e6603740 1 -- 172.21.15.94:0/2892771189 shutdown_connections 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.306+0000 7f94e6603740 1 -- 172.21.15.94:0/2892771189 wait complete. 2024-06-03T01:23:37.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.306+0000 7f94e6603740 1 -- start start 2024-06-03T01:23:37.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.306+0000 7f94e6603740 1 Processor -- start 2024-06-03T01:23:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e6603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b830fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e6603740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b831500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e6603740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b831a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e6603740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562a0b836a70 con 0x562a0b833f80 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e6603740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a0b7480b0 con 0x562a0b82a770 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e6603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a0b836bf0 con 0x562a0b829200 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e6603740 10 client.0 _ll_register_callbacks cb 0x562a0b733f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b830fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b830fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43522/0 (socket says 172.21.15.94:43522) 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e0e00640 1 -- 172.21.15.94:0/1599448271 learned_addr learned my addr 172.21.15.94:0/1599448271 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94d3e00640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b831500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e1800640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b831a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e0e00640 1 -- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 msgr2=0x562a0b831500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.307+0000 7f94e0e00640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b831500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94e0e00640 1 -- 172.21.15.94:0/1599448271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 msgr2=0x562a0b831a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94e0e00640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b831a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94e0e00640 1 -- 172.21.15.94:0/1599448271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0b56d620 con 0x562a0b829200 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94d3e00640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b831500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94e0e00640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b830fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0b8960 tx=0x7f94cc0b8130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8418d048e1f7f1d server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94caa00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94caa00640 1 -- 172.21.15.94:0/1599448271 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0d4070 con 0x562a0b829200 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94cc0bc500 con 0x562a0b829200 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94caa00640 1 -- 172.21.15.94:0/1599448271 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f94cc0c4ce0 con 0x562a0b829200 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.308+0000 7f94caa00640 1 -- 172.21.15.94:0/1599448271 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f94cc0bcc00 con 0x562a0b829200 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 10 client.26368 fetch_fsmap learned FSMap version 64 2024-06-03T01:23:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562a0b836a70 con 0x562a0b829200 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94caa00640 1 -- 172.21.15.94:0/1599448271 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc0bc500 con 0x562a0b829200 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94caa00640 1 -- 172.21.15.94:0/1599448271 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f94cc0d5260 con 0x562a0b829200 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 10 client.26368 fetch_fsmap finished waiting for FSMap version 64 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 20 client.26368 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b843fc0 0x562a0b846440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 4 client.26368 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a0b79ab50 con 0x562a0b843fc0 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.309+0000 7f94d3e00640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b843fc0 0x562a0b846440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.310+0000 7f94caa00640 10 client.26368 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.310+0000 7f94d3e00640 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b843fc0 0x562a0b846440 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d11accf72f2a0 server_cookie=6707e10ed0732b9e in_seq=0 out_seq=0 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94caa00640 1 -- 172.21.15.94:0/1599448271 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a0b79ab50 con 0x562a0b843fc0 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94caa00640 10 client.26368 handle_command_reply: tid=0 2024-06-03T01:23:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 client.26368 shutdown 2024-06-03T01:23:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 msgr2=0x562a0b830fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b830fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0b8960 tx=0x7f94cc0b8130 comp rx=0 tx=0).stop 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 shutdown_connections 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b833f80 0x562a0b831a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b843fc0 0x562a0b846440 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b82a770 0x562a0b831500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 --2- 172.21.15.94:0/1599448271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b830fc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94caa00640 0 client.26368 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 >> 172.21.15.94:0/1599448271 conn(0x562a0b7f1f80 msgr2=0x562a0b82b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 shutdown_connections 2024-06-03T01:23:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.311+0000 7f94e6603740 1 -- 172.21.15.94:0/1599448271 wait complete. 2024-06-03T01:23:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.312+0000 7f94e6603740 20 client.26368 trim_cache size 0 max 16384 2024-06-03T01:23:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.329+0000 7f94e6603740 1 Processor -- start 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e6603740 1 -- start start 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e6603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b8336f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e6603740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b82ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e6603740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b833f80 0x562a0b82b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e6603740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1b5320 con 0x562a0b82a770 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e6603740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1b6f20 con 0x562a0b833f80 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e6603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1b5ab0 con 0x562a0b829200 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b8336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b8336f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43530/0 (socket says 172.21.15.94:43530) 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e0e00640 1 -- 172.21.15.94:0/2999322122 learned_addr learned my addr 172.21.15.94:0/2999322122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b833f80 0x562a0b82b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94d3e00640 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b82ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e0e00640 1 -- 172.21.15.94:0/2999322122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b833f80 msgr2=0x562a0b82b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.330+0000 7f94e0e00640 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b833f80 0x562a0b82b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.331+0000 7f94e0e00640 1 -- 172.21.15.94:0/2999322122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 msgr2=0x562a0b82ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.331+0000 7f94e0e00640 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b82ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.331+0000 7f94e0e00640 1 -- 172.21.15.94:0/2999322122 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0b829cd0 con 0x562a0b829200 2024-06-03T01:23:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.331+0000 7f94e0e00640 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b8336f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0bc200 tx=0x7f94cc0c5170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ce331f5fc3faf2b server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.331+0000 7f94d1600640 1 -- 172.21.15.94:0/2999322122 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0d4070 con 0x562a0b829200 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.331+0000 7f94d1600640 1 -- 172.21.15.94:0/2999322122 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc0bc500 con 0x562a0b829200 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.331+0000 7f94d1600640 1 -- 172.21.15.94:0/2999322122 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004b40 con 0x562a0b829200 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 -- 172.21.15.94:0/2999322122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 msgr2=0x562a0b8336f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b8336f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0bc200 tx=0x7f94cc0c5170 comp rx=0 tx=0).stop 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 -- 172.21.15.94:0/2999322122 shutdown_connections 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b833f80 0x562a0b82b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b82ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 --2- 172.21.15.94:0/2999322122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b829200 0x562a0b8336f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 -- 172.21.15.94:0/2999322122 >> 172.21.15.94:0/2999322122 conn(0x562a0b7f1f80 msgr2=0x562a0b82ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 -- 172.21.15.94:0/2999322122 shutdown_connections 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 -- 172.21.15.94:0/2999322122 wait complete. 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.332+0000 7f94e6603740 1 -- start start 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 1 Processor -- start 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b829200 0x562a0b8a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b8a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b833f80 0x562a0b837400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1b6850 con 0x562a0b82a770 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1b6180 con 0x562a0b829200 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1b3320 con 0x562a0b833f80 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e6603740 10 client.0 _ll_register_callbacks cb 0x7f94cc031440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e1800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b833f80 0x562a0b837400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b8a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e1800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b833f80 0x562a0b837400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:43540/0 (socket says 172.21.15.94:43540) 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.333+0000 7f94e1800640 1 -- 172.21.15.94:0/3971320281 learned_addr learned my addr 172.21.15.94:0/3971320281 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94e0e00640 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b829200 0x562a0b8a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94e1800640 1 -- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b829200 msgr2=0x562a0b8a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94e1800640 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b829200 0x562a0b8a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94e1800640 1 -- 172.21.15.94:0/3971320281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 msgr2=0x562a0b8a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94e1800640 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b8a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94e1800640 1 -- 172.21.15.94:0/3971320281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0b837af0 con 0x562a0b833f80 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94d3e00640 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b8a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94e1800640 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b833f80 0x562a0b837400 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94d400fd80 tx=0x7f94d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d1bc2d73ebb08b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94caa00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:23:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.334+0000 7f94caa00640 1 -- 172.21.15.94:0/3971320281 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d402f070 con 0x562a0b833f80 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94cc0bc500 con 0x562a0b833f80 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94caa00640 1 -- 172.21.15.94:0/3971320281 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f94d402b430 con 0x562a0b833f80 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94caa00640 1 -- 172.21.15.94:0/3971320281 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f94d403b050 con 0x562a0b833f80 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 10 client.26374 fetch_fsmap learned FSMap version 64 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562a0b8ae880 con 0x562a0b833f80 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94caa00640 1 -- 172.21.15.94:0/3971320281 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc0bc500 con 0x562a0b833f80 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94caa00640 1 -- 172.21.15.94:0/3971320281 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f94d401d070 con 0x562a0b833f80 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 10 client.26374 fetch_fsmap finished waiting for FSMap version 64 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 20 client.26374 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b8aefe0 0x562a0b8b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 4 client.26374 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:23:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.335+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a0b794ea0 con 0x562a0b8aefe0 2024-06-03T01:23:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.336+0000 7f94e0e00640 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b8aefe0 0x562a0b8b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.336+0000 7f94e0e00640 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b8aefe0 0x562a0b8b1400 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebfc04d82d6a6b0 server_cookie=31e5f49a18a438ff in_seq=0 out_seq=0 2024-06-03T01:23:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.336+0000 7f94caa00640 10 client.26374 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94caa00640 1 -- 172.21.15.94:0/3971320281 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a0b794ea0 con 0x562a0b8aefe0 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94caa00640 10 client.26374 handle_command_reply: tid=0 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 client.26374 shutdown 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b833f80 msgr2=0x562a0b837400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b833f80 0x562a0b837400 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f94d400fd80 tx=0x7f94d4004730 comp rx=0 tx=0).stop 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 shutdown_connections 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a0b833f80 0x562a0b837400 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0b82a770 0x562a0b8a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x562a0b8aefe0 0x562a0b8b1400 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 --2- 172.21.15.94:0/3971320281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a0b829200 0x562a0b8a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94caa00640 0 client.26374 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.380+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 >> 172.21.15.94:0/3971320281 conn(0x562a0b7f1f80 msgr2=0x562a0b82b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.381+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 shutdown_connections 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.381+0000 7f94e6603740 1 -- 172.21.15.94:0/3971320281 wait complete. 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.381+0000 7f94e6603740 20 client.26374 trim_cache size 0 max 16384 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:23:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:23:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:23:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f94c00a2d70 msgr2=0x7f94c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f94c00a2d70 0x7f94c00a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94d40097d0 tx=0x7f94d4005f90 comp rx=0 tx=0).stop 2024-06-03T01:23:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1017a0 msgr2=0x7f94dc1a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1017a0 0x7f94dc1a4400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009b30 tx=0x7f94cc0305d0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 shutdown_connections 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f94c00a2d70 0x7f94c00a5230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94dc1077b0 0x7f94dc10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94dc102160 0x7f94dc1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 --2- 172.21.15.94:0/1418618321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1017a0 0x7f94dc1a4400 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 >> 172.21.15.94:0/1418618321 conn(0x7f94dc0fd2f0 msgr2=0x7f94dc0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 shutdown_connections 2024-06-03T01:23:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:37.382+0000 7f94e2200640 1 -- 172.21.15.94:0/1418618321 wait complete. 2024-06-03T01:23:38.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 313058 2.19 MB/sec execute 585 sec latency 26.832 ms 2024-06-03T01:23:38.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:38 smithi107 ceph-mon[36622]: pgmap v415: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-03T01:23:38.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:38 smithi094 ceph-mon[32119]: pgmap v415: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-03T01:23:38.604 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:38 smithi179 ceph-mon[37041]: pgmap v415: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-03T01:23:39.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 313847 2.20 MB/sec execute 586 sec latency 230.893 ms 2024-06-03T01:23:39.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:39 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:39.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:39 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:39.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:39 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:40.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 313847 2.19 MB/sec execute 587 sec latency 1230.992 ms 2024-06-03T01:23:40.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:40 smithi107 ceph-mon[36622]: pgmap v416: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-06-03T01:23:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:40 smithi094 ceph-mon[32119]: pgmap v416: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-06-03T01:23:40.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:40 smithi179 ceph-mon[37041]: pgmap v416: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-06-03T01:23:41.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 313981 2.19 MB/sec execute 588 sec latency 2146.849 ms 2024-06-03T01:23:42.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 315350 2.20 MB/sec execute 589 sec latency 20.492 ms 2024-06-03T01:23:42.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:42 smithi107 ceph-mon[36622]: pgmap v417: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-06-03T01:23:42.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:42 smithi094 ceph-mon[32119]: pgmap v417: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-06-03T01:23:42.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:42 smithi179 ceph-mon[37041]: pgmap v417: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-06-03T01:23:43.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 316111 2.21 MB/sec execute 590 sec latency 30.056 ms 2024-06-03T01:23:44.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 316859 2.21 MB/sec execute 591 sec latency 31.435 ms 2024-06-03T01:23:44.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:44 smithi107 ceph-mon[36622]: pgmap v418: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-06-03T01:23:44.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:44 smithi094 ceph-mon[32119]: pgmap v418: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-06-03T01:23:44.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:44 smithi179 ceph-mon[37041]: pgmap v418: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-06-03T01:23:45.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 317396 2.21 MB/sec execute 592 sec latency 485.541 ms 2024-06-03T01:23:46.150 INFO:tasks.workunit.client.0.smithi094.stdout: 1 318597 2.22 MB/sec execute 593 sec latency 659.426 ms 2024-06-03T01:23:46.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:46 smithi107 ceph-mon[36622]: pgmap v419: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-06-03T01:23:46.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:46 smithi094 ceph-mon[32119]: pgmap v419: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-06-03T01:23:46.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:46 smithi179 ceph-mon[37041]: pgmap v419: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-06-03T01:23:47.151 INFO:tasks.workunit.client.0.smithi094.stdout: 1 319534 2.22 MB/sec execute 594 sec latency 18.102 ms 2024-06-03T01:23:48.151 INFO:tasks.workunit.client.0.smithi094.stdout: 1 320160 2.22 MB/sec execute 595 sec latency 44.767 ms 2024-06-03T01:23:48.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:48 smithi107 ceph-mon[36622]: pgmap v420: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-06-03T01:23:48.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:48 smithi094 ceph-mon[32119]: pgmap v420: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-06-03T01:23:48.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:48 smithi179 ceph-mon[37041]: pgmap v420: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-06-03T01:23:49.151 INFO:tasks.workunit.client.0.smithi094.stdout: 1 320938 2.23 MB/sec execute 596 sec latency 303.917 ms 2024-06-03T01:23:50.151 INFO:tasks.workunit.client.0.smithi094.stdout: 1 320938 2.22 MB/sec execute 597 sec latency 1304.094 ms 2024-06-03T01:23:50.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:50 smithi107 ceph-mon[36622]: pgmap v421: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-06-03T01:23:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 -- 172.21.15.94:0/3739360826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c680ffca0 msgr2=0x7f7c68100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 --2- 172.21.15.94:0/3739360826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c680ffca0 0x7f7c68100100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009900 tx=0x7f7c5402a480 comp rx=0 tx=0).stop 2024-06-03T01:23:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 -- 172.21.15.94:0/3739360826 shutdown_connections 2024-06-03T01:23:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 --2- 172.21.15.94:0/3739360826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c68100640 0x7f7c681053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 --2- 172.21.15.94:0/3739360826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c680ffca0 0x7f7c68100100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 --2- 172.21.15.94:0/3739360826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c68106700 0x7f7c68106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 -- 172.21.15.94:0/3739360826 >> 172.21.15.94:0/3739360826 conn(0x7f7c68076c20 msgr2=0x7f7c680fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 -- 172.21.15.94:0/3739360826 shutdown_connections 2024-06-03T01:23:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.536+0000 7f7c67200640 1 -- 172.21.15.94:0/3739360826 wait complete. 2024-06-03T01:23:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.537+0000 7f7c67200640 1 Processor -- start 2024-06-03T01:23:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.537+0000 7f7c67200640 1 -- start start 2024-06-03T01:23:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c67200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c680ffca0 0x7f7c6819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c67200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c68100640 0x7f7c6819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c67200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c68106700 0x7f7c681a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c67200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68118180 con 0x7f7c68100640 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c67200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68118000 con 0x7f7c68106700 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c67200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68118300 con 0x7f7c680ffca0 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c680ffca0 0x7f7c6819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c680ffca0 0x7f7c6819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:59218/0 (socket says 172.21.15.94:59218) 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c65400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c68100640 0x7f7c6819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c65e00640 1 -- 172.21.15.94:0/332303928 learned_addr learned my addr 172.21.15.94:0/332303928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.538+0000 7f7c66800640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c68106700 0x7f7c681a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c65e00640 1 -- 172.21.15.94:0/332303928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c68106700 msgr2=0x7f7c681a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c65e00640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c68106700 0x7f7c681a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c65e00640 1 -- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c68100640 msgr2=0x7f7c6819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c65e00640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c68100640 0x7f7c6819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c65e00640 1 -- 172.21.15.94:0/332303928 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c681a3660 con 0x7f7c680ffca0 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c65400640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c68100640 0x7f7c6819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c66800640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c68106700 0x7f7c681a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c65e00640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c680ffca0 0x7f7c6819e6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800e9f0 tx=0x7f7c5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c5ea00640 1 -- 172.21.15.94:0/332303928 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800cd80 con 0x7f7c680ffca0 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c681a3950 con 0x7f7c680ffca0 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c5ea00640 1 -- 172.21.15.94:0/332303928 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c58010910 con 0x7f7c680ffca0 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.539+0000 7f7c5ea00640 1 -- 172.21.15.94:0/332303928 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580188f0 con 0x7f7c680ffca0 2024-06-03T01:23:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.540+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681ab1e0 con 0x7f7c680ffca0 2024-06-03T01:23:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.543+0000 7f7c6f837740 1 Processor -- start 2024-06-03T01:23:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.543+0000 7f7c5ea00640 1 -- 172.21.15.94:0/332303928 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7c58010ac0 con 0x7f7c680ffca0 2024-06-03T01:23:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.543+0000 7f7c6f837740 1 -- start start 2024-06-03T01:23:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e07cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e08680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e15370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c6f837740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561dc8d6c420 con 0x561dc8e12f80 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c6f837740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561dc8d13c50 con 0x561dc8e08200 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c6f837740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561dc8d683e0 con 0x561dc8e09770 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c66800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e07cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e08680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48792/0 (socket says 172.21.15.94:48792) 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 -- 172.21.15.94:0/973454099 learned_addr learned my addr 172.21.15.94:0/973454099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65400640 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e15370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 -- 172.21.15.94:0/973454099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 msgr2=0x561dc8e07cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e07cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 -- 172.21.15.94:0/973454099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 msgr2=0x561dc8e15370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e15370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c65e00640 1 -- 172.21.15.94:0/973454099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc8a80210 con 0x561dc8e08200 2024-06-03T01:23:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.544+0000 7f7c66800640 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e07cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.545+0000 7f7c65e00640 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e08680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7c580d3d20 tx=0x7f7c580d8670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ec1682b1c3f15e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.545+0000 7f7c5ea00640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7c4c0a2cf0 0x7f7c4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.545+0000 7f7c53000640 1 -- 172.21.15.94:0/973454099 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580df020 con 0x561dc8e08200 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.545+0000 7f7c53000640 1 -- 172.21.15.94:0/973454099 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5809bd60 con 0x561dc8e08200 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.545+0000 7f7c53000640 1 -- 172.21.15.94:0/973454099 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580c5d60 con 0x561dc8e08200 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.545+0000 7f7c5ea00640 1 -- 172.21.15.94:0/332303928 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f7c580cfbb0 con 0x7f7c680ffca0 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.545+0000 7f7c66800640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7c4c0a2cf0 0x7f7c4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 -- 172.21.15.94:0/973454099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 msgr2=0x561dc8e08680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e08680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7c580d3d20 tx=0x7f7c580d8670 comp rx=0 tx=0).stop 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c66800640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7c4c0a2cf0 0x7f7c4c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009fd0 tx=0x7f7c60009500 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 -- 172.21.15.94:0/973454099 shutdown_connections 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e15370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e08680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 --2- 172.21.15.94:0/973454099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e07cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 -- 172.21.15.94:0/973454099 >> 172.21.15.94:0/973454099 conn(0x561dc8dd0f80 msgr2=0x561dc8e047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 -- 172.21.15.94:0/973454099 shutdown_connections 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.546+0000 7f7c6f837740 1 -- 172.21.15.94:0/973454099 wait complete. 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 -- start start 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 Processor -- start 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e08200 0x561dc8e0ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e10460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e109a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x561dc8e15cf0 con 0x561dc8e12f80 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561dc8d270b0 con 0x561dc8e09770 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.547+0000 7f7c6f837740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561dc8e15e70 con 0x561dc8e08200 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c6f837740 10 client.0 _ll_register_callbacks cb 0x561dc8d12f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e10460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e08200 0x561dc8e0ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e08200 0x561dc8e0ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:59236/0 (socket says 172.21.15.94:59236) 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e10460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48808/0 (socket says 172.21.15.94:48808) 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 -- 172.21.15.94:0/1204561272 learned_addr learned my addr 172.21.15.94:0/1204561272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c66800640 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 -- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 msgr2=0x561dc8e10460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e10460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 -- 172.21.15.94:0/1204561272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 msgr2=0x561dc8e109a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e109a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 -- 172.21.15.94:0/1204561272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc8b4c620 con 0x561dc8e08200 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65400640 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e10460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c65e00640 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e08200 0x561dc8e0ff20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c5809bcf0 tx=0x7f7c580d8f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c90231d63a39429 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.548+0000 7f7c47400640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c47400640 1 -- 172.21.15.94:0/1204561272 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580c64b0 con 0x561dc8e08200 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c47400640 1 -- 172.21.15.94:0/1204561272 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f7c580d3860 con 0x561dc8e08200 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c5809bd60 con 0x561dc8e08200 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c47400640 1 -- 172.21.15.94:0/1204561272 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f7c580df030 con 0x561dc8e08200 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c6f837740 10 client.26386 fetch_fsmap learned FSMap version 65 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x561dc8e15cf0 con 0x561dc8e08200 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c47400640 1 -- 172.21.15.94:0/1204561272 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5809bd60 con 0x561dc8e08200 2024-06-03T01:23:50.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.549+0000 7f7c47400640 1 -- 172.21.15.94:0/1204561272 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 65) ==== 8378+0+0 (secure 0 0 0) 0x7f7c580c76c0 con 0x561dc8e08200 2024-06-03T01:23:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c6f837740 10 client.26386 fetch_fsmap finished waiting for FSMap version 65 2024-06-03T01:23:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c6f837740 10 client.26386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c6f837740 20 client.26386 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e23170 0x561dc8e25560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c6f837740 4 client.26386 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:23:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc8d79b50 con 0x561dc8e23170 2024-06-03T01:23:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c65400640 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e23170 0x561dc8e25560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c65400640 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e23170 0x561dc8e25560 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd98633248d253c4 server_cookie=2cc58cfb8840246c in_seq=0 out_seq=0 2024-06-03T01:23:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.550+0000 7f7c47400640 10 client.26386 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c47400640 1 -- 172.21.15.94:0/1204561272 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561dc8d79b50 con 0x561dc8e23170 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c47400640 10 client.26386 handle_command_reply: tid=0 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 client.26386 shutdown 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e08200 msgr2=0x561dc8e0ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e08200 0x561dc8e0ff20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c5809bcf0 tx=0x7f7c580d8f90 comp rx=0 tx=0).stop 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 shutdown_connections 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e23170 0x561dc8e25560 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e10460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1204561272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e08200 0x561dc8e0ff20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c47400640 0 client.26386 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 >> 172.21.15.94:0/1204561272 conn(0x561dc8dd0f80 msgr2=0x561dc8e0a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.552+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 shutdown_connections 2024-06-03T01:23:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.553+0000 7f7c6f837740 1 -- 172.21.15.94:0/1204561272 wait complete. 2024-06-03T01:23:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.553+0000 7f7c6f837740 20 client.26386 trim_cache size 0 max 16384 2024-06-03T01:23:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.570+0000 7f7c6f837740 1 Processor -- start 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c6f837740 1 -- start start 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e2b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c6f837740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b5390 con 0x561dc8e12f80 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c6f837740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b6f90 con 0x561dc8e08200 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c6f837740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b5b20 con 0x561dc8e09770 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c66800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e2b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c66800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e2b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44586/0 (socket says 172.21.15.94:44586) 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c66800640 1 -- 172.21.15.94:0/2695505253 learned_addr learned my addr 172.21.15.94:0/2695505253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:50.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.571+0000 7f7c65400640 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c66800640 1 -- 172.21.15.94:0/2695505253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 msgr2=0x561dc8e2acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c66800640 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e2acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c66800640 1 -- 172.21.15.94:0/2695505253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 msgr2=0x561dc8e037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c66800640 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c66800640 1 -- 172.21.15.94:0/2695505253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc8e0e470 con 0x561dc8e12f80 2024-06-03T01:23:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c66800640 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e2b210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7c600113a0 tx=0x7f7c60016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249e51361952c478 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:50.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c52600640 1 -- 172.21.15.94:0/2695505253 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60011cb0 con 0x561dc8e12f80 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c52600640 1 -- 172.21.15.94:0/2695505253 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c60011e50 con 0x561dc8e12f80 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.572+0000 7f7c52600640 1 -- 172.21.15.94:0/2695505253 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60010630 con 0x561dc8e12f80 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 -- 172.21.15.94:0/2695505253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 msgr2=0x561dc8e2b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e2b210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7c600113a0 tx=0x7f7c60016000 comp rx=0 tx=0).stop 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 -- 172.21.15.94:0/2695505253 shutdown_connections 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e12f80 0x561dc8e2b210 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e09770 0x561dc8e2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 --2- 172.21.15.94:0/2695505253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e08200 0x561dc8e037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 -- 172.21.15.94:0/2695505253 >> 172.21.15.94:0/2695505253 conn(0x561dc8dd0f80 msgr2=0x561dc8e030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:50.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 -- 172.21.15.94:0/2695505253 shutdown_connections 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.573+0000 7f7c6f837740 1 -- 172.21.15.94:0/2695505253 wait complete. 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.574+0000 7f7c6f837740 1 -- start start 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.574+0000 7f7c6f837740 1 Processor -- start 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.574+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e08200 0x561dc8e1b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.574+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e0cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c6f837740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e12f80 0x561dc8e0d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c6f837740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b68c0 con 0x561dc8e08200 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c6f837740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b61f0 con 0x561dc8e09770 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c6f837740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b3390 con 0x561dc8e12f80 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c6f837740 10 client.0 _ll_register_callbacks cb 0x7f7c58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e0cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e08200 0x561dc8e1b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c66800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e12f80 0x561dc8e0d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e0cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48812/0 (socket says 172.21.15.94:48812) 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 -- 172.21.15.94:0/1014682920 learned_addr learned my addr 172.21.15.94:0/1014682920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 -- 172.21.15.94:0/1014682920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e12f80 msgr2=0x561dc8e0d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e12f80 0x561dc8e0d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 -- 172.21.15.94:0/1014682920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e08200 msgr2=0x561dc8e1b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e08200 0x561dc8e1b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 -- 172.21.15.94:0/1014682920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc8e0d9c0 con 0x561dc8e09770 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c65400640 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e0cc50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009900 tx=0x7f7c54005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa8a5227343d961 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.575+0000 7f7c47400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c47400640 1 -- 172.21.15.94:0/1014682920 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54035040 con 0x561dc8e09770 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c60011e50 con 0x561dc8e09770 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c47400640 1 -- 172.21.15.94:0/1014682920 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f7c54004760 con 0x561dc8e09770 2024-06-03T01:23:50.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c47400640 1 -- 172.21.15.94:0/1014682920 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f7c54056050 con 0x561dc8e09770 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 10 client.26378 fetch_fsmap learned FSMap version 65 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x561dc8e8dcf0 con 0x561dc8e09770 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c47400640 1 -- 172.21.15.94:0/1014682920 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c60011e50 con 0x561dc8e09770 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c47400640 1 -- 172.21.15.94:0/1014682920 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 65) ==== 8378+0+0 (secure 0 0 0) 0x7f7c54031ce0 con 0x561dc8e09770 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 10 client.26378 fetch_fsmap finished waiting for FSMap version 65 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 20 client.26378 populate_metadata read hostname 'smithi094' 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e8e470 0x561dc8e90890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 4 client.26378 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.576+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dc8d73ea0 con 0x561dc8e8e470 2024-06-03T01:23:50.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.577+0000 7f7c65e00640 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e8e470 0x561dc8e90890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:23:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.577+0000 7f7c65e00640 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e8e470 0x561dc8e90890 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875909e9afdb70 server_cookie=488210a884205b2e in_seq=0 out_seq=0 2024-06-03T01:23:50.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.577+0000 7f7c47400640 10 client.26378 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:50.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:50 smithi094 ceph-mon[32119]: pgmap v421: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-06-03T01:23:50.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:50 smithi179 ceph-mon[37041]: pgmap v421: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-06-03T01:23:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c47400640 1 -- 172.21.15.94:0/1014682920 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561dc8d73ea0 con 0x561dc8e8e470 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c47400640 10 client.26378 handle_command_reply: tid=0 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 client.26378 shutdown 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 msgr2=0x561dc8e0cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e0cc50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009900 tx=0x7f7c54005250 comp rx=0 tx=0).stop 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 shutdown_connections 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561dc8e12f80 0x561dc8e0d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x561dc8e8e470 0x561dc8e90890 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561dc8e09770 0x561dc8e0cc50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c47400640 0 client.26378 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 --2- 172.21.15.94:0/1014682920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561dc8e08200 0x561dc8e1b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.614 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.613+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 >> 172.21.15.94:0/1014682920 conn(0x561dc8dd0f80 msgr2=0x561dc8e09c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.614+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 shutdown_connections 2024-06-03T01:23:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.614+0000 7f7c6f837740 1 -- 172.21.15.94:0/1014682920 wait complete. 2024-06-03T01:23:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.614+0000 7f7c6f837740 20 client.26378 trim_cache size 0 max 16384 2024-06-03T01:23:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:23:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:23:50.615 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:23:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.614+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7c4c0a2cf0 msgr2=0x7f7c4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.614+0000 7f7c67200640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7c4c0a2cf0 0x7f7c4c0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009fd0 tx=0x7f7c60009500 comp rx=0 tx=0).stop 2024-06-03T01:23:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.614+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c680ffca0 msgr2=0x7f7c6819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:23:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.614+0000 7f7c67200640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c680ffca0 0x7f7c6819e6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800e9f0 tx=0x7f7c5800eeb0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 shutdown_connections 2024-06-03T01:23:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f7c4c0a2cf0 0x7f7c4c0a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c68106700 0x7f7c681a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c68100640 0x7f7c6819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 --2- 172.21.15.94:0/332303928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c680ffca0 0x7f7c6819e6c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:23:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 >> 172.21.15.94:0/332303928 conn(0x7f7c68076c20 msgr2=0x7f7c681033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:23:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 shutdown_connections 2024-06-03T01:23:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:23:50.615+0000 7f7c67200640 1 -- 172.21.15.94:0/332303928 wait complete. 2024-06-03T01:23:51.151 INFO:tasks.workunit.client.0.smithi094.stdout: 1 322176 2.23 MB/sec execute 598 sec latency 1478.031 ms 2024-06-03T01:23:52.151 INFO:tasks.workunit.client.0.smithi094.stdout: 1 323156 2.24 MB/sec execute 599 sec latency 16.121 ms 2024-06-03T01:23:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:52 smithi107 ceph-mon[36622]: pgmap v422: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-06-03T01:23:52.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:52 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:52 smithi094 ceph-mon[32119]: pgmap v422: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-06-03T01:23:52.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:52 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:52 smithi179 ceph-mon[37041]: pgmap v422: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-06-03T01:23:52.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:52 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:23:53.151 INFO:tasks.workunit.client.0.smithi094.stdout: 1 cleanup 600 sec 2024-06-03T01:23:53.399 INFO:tasks.workunit.client.0.smithi094.stdout: 0 cleanup 600 sec 2024-06-03T01:23:53.399 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-06-03T01:23:53.399 INFO:tasks.workunit.client.0.smithi094.stdout: Operation Count AvgLat MaxLat 2024-06-03T01:23:53.399 INFO:tasks.workunit.client.0.smithi094.stdout: ---------------------------------------- 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: NTCreateX 43200 8.073 5606.803 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Close 31691 0.011 0.388 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Rename 1831 53.637 4577.029 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Unlink 8761 4.474 7589.836 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Qpathinfo 39154 1.598 58.537 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Qfileinfo 6808 0.005 0.104 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Qfsinfo 7187 0.492 32.062 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Sfileinfo 3525 0.116 9.874 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: Find 15115 0.462 34.186 2024-06-03T01:23:53.400 INFO:tasks.workunit.client.0.smithi094.stdout: WriteX 21322 0.589 113.102 2024-06-03T01:23:53.401 INFO:tasks.workunit.client.0.smithi094.stdout: ReadX 67544 0.061 494.197 2024-06-03T01:23:53.401 INFO:tasks.workunit.client.0.smithi094.stdout: LockX 140 0.982 2.615 2024-06-03T01:23:53.401 INFO:tasks.workunit.client.0.smithi094.stdout: UnlockX 140 0.857 3.273 2024-06-03T01:23:53.401 INFO:tasks.workunit.client.0.smithi094.stdout: Flush 3034 6.909 51.310 2024-06-03T01:23:53.401 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-06-03T01:23:53.401 INFO:tasks.workunit.client.0.smithi094.stdout:Throughput 2.23713 MB/sec 1 clients 1 procs max_latency=7589.858 ms 2024-06-03T01:23:53.401 INFO:tasks.workunit.client.0.smithi094.stderr:+ dbench 10 2024-06-03T01:23:53.402 INFO:tasks.workunit.client.0.smithi094.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-03T01:23:53.402 INFO:tasks.workunit.client.0.smithi094.stdout: 2024-06-03T01:23:53.402 INFO:tasks.workunit.client.0.smithi094.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-03T01:23:53.406 INFO:tasks.workunit.client.0.smithi094.stdout:9 of 10 processes prepared for launch 0 sec 2024-06-03T01:23:53.506 INFO:tasks.workunit.client.0.smithi094.stdout:10 of 10 processes prepared for launch 0 sec 2024-06-03T01:23:53.506 INFO:tasks.workunit.client.0.smithi094.stdout:releasing clients 2024-06-03T01:23:54.506 INFO:tasks.workunit.client.0.smithi094.stdout: 10 402 142.66 MB/sec warmup 1 sec latency 64.373 ms 2024-06-03T01:23:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:54 smithi107 ceph-mon[36622]: pgmap v423: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-06-03T01:23:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:54 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:54 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:54.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:54 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:23:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:54 smithi094 ceph-mon[32119]: pgmap v423: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-06-03T01:23:54.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:54 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:54 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:54.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:54 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:23:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:54 smithi179 ceph-mon[37041]: pgmap v423: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 193 op/s 2024-06-03T01:23:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:54 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:23:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:54 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:23:54.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:54 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:23:55.506 INFO:tasks.workunit.client.0.smithi094.stdout: 10 864 103.94 MB/sec warmup 2 sec latency 545.956 ms 2024-06-03T01:23:56.506 INFO:tasks.workunit.client.0.smithi094.stdout: 10 1406 82.32 MB/sec warmup 3 sec latency 1546.046 ms 2024-06-03T01:23:56.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:56 smithi107 ceph-mon[36622]: pgmap v424: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-06-03T01:23:56.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:56 smithi094 ceph-mon[32119]: pgmap v424: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-06-03T01:23:56.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:56 smithi179 ceph-mon[37041]: pgmap v424: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-06-03T01:23:57.506 INFO:tasks.workunit.client.0.smithi094.stdout: 10 2219 66.39 MB/sec warmup 4 sec latency 2474.532 ms 2024-06-03T01:23:57.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:57 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:57.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:57 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:57.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:57 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:58.506 INFO:tasks.workunit.client.0.smithi094.stdout: 10 2754 63.48 MB/sec warmup 5 sec latency 127.031 ms 2024-06-03T01:23:58.532 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:58 smithi107 ceph-mon[36622]: pgmap v425: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-06-03T01:23:58.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:58 smithi094 ceph-mon[32119]: pgmap v425: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-06-03T01:23:58.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:58 smithi179 ceph-mon[37041]: pgmap v425: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-06-03T01:23:59.506 INFO:tasks.workunit.client.0.smithi094.stdout: 10 3391 61.80 MB/sec warmup 6 sec latency 136.323 ms 2024-06-03T01:23:59.598 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:23:59 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:59.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:23:59 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:23:59.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:23:59 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:00.507 INFO:tasks.workunit.client.0.smithi094.stdout: 10 3613 54.45 MB/sec warmup 7 sec latency 76.857 ms 2024-06-03T01:24:00.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:00 smithi094 ceph-mon[32119]: pgmap v426: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-06-03T01:24:00.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:00 smithi179 ceph-mon[37041]: pgmap v426: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-06-03T01:24:00.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:00 smithi107 ceph-mon[36622]: pgmap v426: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-06-03T01:24:01.507 INFO:tasks.workunit.client.0.smithi094.stdout: 10 3778 47.95 MB/sec warmup 8 sec latency 82.269 ms 2024-06-03T01:24:02.507 INFO:tasks.workunit.client.0.smithi094.stdout: 10 3971 42.90 MB/sec warmup 9 sec latency 244.296 ms 2024-06-03T01:24:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:02 smithi094 ceph-mon[32119]: pgmap v427: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-06-03T01:24:02.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:02 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:02 smithi179 ceph-mon[37041]: pgmap v427: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-06-03T01:24:02.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:02 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:02 smithi107 ceph-mon[36622]: pgmap v427: 129 pgs: 129 active+clean; 132 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-06-03T01:24:02.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:02 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:03.507 INFO:tasks.workunit.client.0.smithi094.stdout: 10 4247 39.54 MB/sec warmup 10 sec latency 188.394 ms 2024-06-03T01:24:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.804+0000 7f93f2200640 1 -- 172.21.15.94:0/2767696000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 msgr2=0x7f93ec1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.804+0000 7f93f2200640 1 --2- 172.21.15.94:0/2767696000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 0x7f93ec1041d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009900 tx=0x7f93dc02a3f0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.805+0000 7f93f2200640 1 -- 172.21.15.94:0/2767696000 shutdown_connections 2024-06-03T01:24:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.805+0000 7f93f2200640 1 --2- 172.21.15.94:0/2767696000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ec108ec0 0x7f93ec110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.805+0000 7f93f2200640 1 --2- 172.21.15.94:0/2767696000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec1047a0 0x7f93ec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.805+0000 7f93f2200640 1 --2- 172.21.15.94:0/2767696000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 0x7f93ec1041d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.805+0000 7f93f2200640 1 -- 172.21.15.94:0/2767696000 >> 172.21.15.94:0/2767696000 conn(0x7f93ec0ff690 msgr2=0x7f93ec101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.805+0000 7f93f2200640 1 -- 172.21.15.94:0/2767696000 shutdown_connections 2024-06-03T01:24:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.805+0000 7f93f2200640 1 -- 172.21.15.94:0/2767696000 wait complete. 2024-06-03T01:24:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.806+0000 7f93f2200640 1 Processor -- start 2024-06-03T01:24:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.807+0000 7f93f2200640 1 -- start start 2024-06-03T01:24:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.807+0000 7f93f2200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 0x7f93ec19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.807+0000 7f93f2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ec108ec0 0x7f93ec19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.807+0000 7f93f2200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec19ec70 0x7f93ec19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.807+0000 7f93f2200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec113fc0 con 0x7f93ec19ec70 2024-06-03T01:24:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.807+0000 7f93f2200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec113e40 con 0x7f93ec108ec0 2024-06-03T01:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.807+0000 7f93f2200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec114140 con 0x7f93ec103df0 2024-06-03T01:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93eaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ec108ec0 0x7f93ec19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec19ec70 0x7f93ec19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93eb400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 0x7f93ec19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93eaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ec108ec0 0x7f93ec19a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37254/0 (socket says 172.21.15.94:37254) 2024-06-03T01:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93eaa00640 1 -- 172.21.15.94:0/1611988883 learned_addr learned my addr 172.21.15.94:0/1611988883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec19ec70 0x7f93ec19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53702/0 (socket says 172.21.15.94:53702) 2024-06-03T01:24:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93ebe00640 1 -- 172.21.15.94:0/1611988883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 msgr2=0x7f93ec19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:03.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.808+0000 7f93ebe00640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 0x7f93ec19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.809+0000 7f93ebe00640 1 -- 172.21.15.94:0/1611988883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ec108ec0 msgr2=0x7f93ec19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.809+0000 7f93ebe00640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ec108ec0 0x7f93ec19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.809+0000 7f93ebe00640 1 -- 172.21.15.94:0/1611988883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec19f760 con 0x7f93ec19ec70 2024-06-03T01:24:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.809+0000 7f93eb400640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 0x7f93ec19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:24:03.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.809+0000 7f93ebe00640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec19ec70 0x7f93ec19f120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ec50 tx=0x7f93e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.810+0000 7f93e3e00640 1 -- 172.21.15.94:0/1611988883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4003b70 con 0x7f93ec19ec70 2024-06-03T01:24:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.810+0000 7f93e3e00640 1 -- 172.21.15.94:0/1611988883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e4003d10 con 0x7f93ec19ec70 2024-06-03T01:24:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.810+0000 7f93e3e00640 1 -- 172.21.15.94:0/1611988883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4010890 con 0x7f93ec19ec70 2024-06-03T01:24:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.810+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec19fa50 con 0x7f93ec19ec70 2024-06-03T01:24:03.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.811+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec1a71f0 con 0x7f93ec19ec70 2024-06-03T01:24:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.817+0000 7f93e3e00640 1 -- 172.21.15.94:0/1611988883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f93e4003fe0 con 0x7f93ec19ec70 2024-06-03T01:24:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 Processor -- start 2024-06-03T01:24:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 -- start start 2024-06-03T01:24:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 0x5627d4e0fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5627d4d74420 con 0x5627d4e115b0 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5627d4d1bc50 con 0x5627d4e235d0 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.819+0000 7f93f5074740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5627d4d703e0 con 0x5627d4e10040 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e104c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36082/0 (socket says 172.21.15.94:36082) 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 -- 172.21.15.94:0/866230306 learned_addr learned my addr 172.21.15.94:0/866230306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eaa00640 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 -- 172.21.15.94:0/866230306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 msgr2=0x5627d4e259c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e259c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 -- 172.21.15.94:0/866230306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 msgr2=0x5627d4e0fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 0x5627d4e0fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.820+0000 7f93eb400640 1 -- 172.21.15.94:0/866230306 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d4a88210 con 0x5627d4e10040 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.821+0000 7f93eb400640 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e104c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009900 tx=0x7f93dc004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720c978de9333647 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.822+0000 7f93d7e00640 1 -- 172.21.15.94:0/866230306 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc03f070 con 0x5627d4e10040 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.822+0000 7f93d7e00640 1 -- 172.21.15.94:0/866230306 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93dc005530 con 0x5627d4e10040 2024-06-03T01:24:03.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.822+0000 7f93d7e00640 1 -- 172.21.15.94:0/866230306 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc035070 con 0x5627d4e10040 2024-06-03T01:24:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.822+0000 7f93f5074740 1 -- 172.21.15.94:0/866230306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 msgr2=0x5627d4e104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.822+0000 7f93f5074740 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e104c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009900 tx=0x7f93dc004150 comp rx=0 tx=0).stop 2024-06-03T01:24:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.823+0000 7f93f5074740 1 -- 172.21.15.94:0/866230306 shutdown_connections 2024-06-03T01:24:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.823+0000 7f93f5074740 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e259c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.823+0000 7f93f5074740 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e104c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.823+0000 7f93f5074740 1 --2- 172.21.15.94:0/866230306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 0x5627d4e0fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.823+0000 7f93f5074740 1 -- 172.21.15.94:0/866230306 >> 172.21.15.94:0/866230306 conn(0x5627d4dd8f80 msgr2=0x5627d4e0c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.823+0000 7f93f5074740 1 -- 172.21.15.94:0/866230306 shutdown_connections 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.824+0000 7f93f5074740 1 -- 172.21.15.94:0/866230306 wait complete. 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.824+0000 7f93e3e00640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f93d00a2e90 0x7f93d00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.824+0000 7f93f5074740 1 -- start start 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.824+0000 7f93e3e00640 1 -- 172.21.15.94:0/1611988883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f93e40cfd40 con 0x7f93ec19ec70 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 1 Processor -- start 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e20640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e115b0 0x5627d4e20b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e235d0 0x5627d4e210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5627d4e26340 con 0x5627d4e235d0 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5627d4d2f0b0 con 0x5627d4e115b0 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5627d4e264c0 con 0x5627d4e10040 2024-06-03T01:24:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.825+0000 7f93f5074740 10 client.0 _ll_register_callbacks cb 0x5627d4d1af40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e20640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e20640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36088/0 (socket says 172.21.15.94:36088) 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 -- 172.21.15.94:0/1893560275 learned_addr learned my addr 172.21.15.94:0/1893560275 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 -- 172.21.15.94:0/1893560275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e115b0 msgr2=0x5627d4e20b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e115b0 0x5627d4e20b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 -- 172.21.15.94:0/1893560275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e235d0 msgr2=0x5627d4e210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e235d0 0x5627d4e210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.826+0000 7f93eb400640 1 -- 172.21.15.94:0/1893560275 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d4b54620 con 0x5627d4e10040 2024-06-03T01:24:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.827+0000 7f93eaa00640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f93d00a2e90 0x7f93d00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.827+0000 7f93eb400640 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e20640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0092a0 tx=0x7f93dc004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74352920cb6dd4e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.827+0000 7f93d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.827+0000 7f93d4c00640 1 -- 172.21.15.94:0/1893560275 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc03f070 con 0x5627d4e10040 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.827+0000 7f93d4c00640 1 -- 172.21.15.94:0/1893560275 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f93dc0315b0 con 0x5627d4e10040 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.828+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93dc005530 con 0x5627d4e10040 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.828+0000 7f93eaa00640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f93d00a2e90 0x7f93d00a5350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5627d4e21c10 tx=0x7f93d8012040 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.828+0000 7f93d4c00640 1 -- 172.21.15.94:0/1893560275 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f93dc035040 con 0x5627d4e10040 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.828+0000 7f93f5074740 10 client.26404 fetch_fsmap learned FSMap version 68 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.828+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5627d4e26340 con 0x5627d4e10040 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.829+0000 7f93d4c00640 1 -- 172.21.15.94:0/1893560275 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93dc005530 con 0x5627d4e10040 2024-06-03T01:24:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.830+0000 7f93d4c00640 1 -- 172.21.15.94:0/1893560275 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 68) ==== 8398+0+0 (secure 0 0 0) 0x7f93dc003a50 con 0x5627d4e10040 2024-06-03T01:24:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.830+0000 7f93f5074740 10 client.26404 fetch_fsmap finished waiting for FSMap version 68 2024-06-03T01:24:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.830+0000 7f93f5074740 10 client.26404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:24:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.830+0000 7f93f5074740 20 client.26404 populate_metadata read hostname 'smithi094' 2024-06-03T01:24:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.830+0000 7f93f5074740 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e338b0 0x5627d4e35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.830+0000 7f93f5074740 4 client.26404 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-03T01:24:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.830+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d4d81b50 con 0x5627d4e338b0 2024-06-03T01:24:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.831+0000 7f93eaa00640 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e338b0 0x5627d4e35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.833+0000 7f93eaa00640 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e338b0 0x5627d4e35ca0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92aa260576a63013 server_cookie=aa1dc6dee9625526 in_seq=0 out_seq=0 2024-06-03T01:24:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.833+0000 7f93d4c00640 10 client.26404 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:24:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.834+0000 7f93d4c00640 1 -- 172.21.15.94:0/1893560275 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5627d4d81b50 con 0x5627d4e338b0 2024-06-03T01:24:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.834+0000 7f93d4c00640 10 client.26404 handle_command_reply: tid=0 2024-06-03T01:24:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.834+0000 7f93f5074740 1 client.26404 shutdown 2024-06-03T01:24:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 msgr2=0x5627d4e20640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e20640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0092a0 tx=0x7f93dc004a70 comp rx=0 tx=0).stop 2024-06-03T01:24:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 shutdown_connections 2024-06-03T01:24:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e235d0 0x5627d4e210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e338b0 0x5627d4e35ca0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e115b0 0x5627d4e20b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 --2- 172.21.15.94:0/1893560275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e10040 0x5627d4e20640 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 >> 172.21.15.94:0/1893560275 conn(0x5627d4dd8f80 msgr2=0x5627d4e124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.835+0000 7f93d4c00640 0 client.26404 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:24:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.836+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 shutdown_connections 2024-06-03T01:24:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.836+0000 7f93f5074740 1 -- 172.21.15.94:0/1893560275 wait complete. 2024-06-03T01:24:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.836+0000 7f93f5074740 20 client.26404 trim_cache size 0 max 16384 2024-06-03T01:24:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.872+0000 7f93f5074740 1 Processor -- start 2024-06-03T01:24:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93f5074740 1 -- start start 2024-06-03T01:24:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93f5074740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e10040 0x5627d4e0b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93f5074740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 0x5627d4e249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93f5074740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e235d0 0x5627d4e24f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93f5074740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1aca40 con 0x5627d4e115b0 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93f5074740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1ae700 con 0x5627d4e10040 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93f5074740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1ad1d0 con 0x5627d4e235d0 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e235d0 0x5627d4e24f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e235d0 0x5627d4e24f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36090/0 (socket says 172.21.15.94:36090) 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.873+0000 7f93ebe00640 1 -- 172.21.15.94:0/4099683974 learned_addr learned my addr 172.21.15.94:0/4099683974 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.874+0000 7f93eaa00640 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 0x5627d4e249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.874+0000 7f93ebe00640 1 -- 172.21.15.94:0/4099683974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e10040 msgr2=0x5627d4e0b690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.874+0000 7f93ebe00640 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e10040 0x5627d4e0b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.874+0000 7f93ebe00640 1 -- 172.21.15.94:0/4099683974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 msgr2=0x5627d4e249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.874+0000 7f93ebe00640 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 0x5627d4e249c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.874+0000 7f93ebe00640 1 -- 172.21.15.94:0/4099683974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d4e25ef0 con 0x5627d4e235d0 2024-06-03T01:24:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.874+0000 7f93ebe00640 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e235d0 0x5627d4e24f00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93e409bdd0 tx=0x7f93e40981e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a127ca0df1fbc1e server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.875+0000 7f93d7400640 1 -- 172.21.15.94:0/4099683974 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e409e070 con 0x5627d4e235d0 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.875+0000 7f93d7400640 1 -- 172.21.15.94:0/4099683974 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e4009820 con 0x5627d4e235d0 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.875+0000 7f93d7400640 1 -- 172.21.15.94:0/4099683974 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4098610 con 0x5627d4e235d0 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 -- 172.21.15.94:0/4099683974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e235d0 msgr2=0x5627d4e24f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e235d0 0x5627d4e24f00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f93e409bdd0 tx=0x7f93e40981e0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 -- 172.21.15.94:0/4099683974 shutdown_connections 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e235d0 0x5627d4e24f00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e115b0 0x5627d4e249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 --2- 172.21.15.94:0/4099683974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e10040 0x5627d4e0b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 -- 172.21.15.94:0/4099683974 >> 172.21.15.94:0/4099683974 conn(0x5627d4dd8f80 msgr2=0x5627d4e0af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 -- 172.21.15.94:0/4099683974 shutdown_connections 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.876+0000 7f93f5074740 1 -- 172.21.15.94:0/4099683974 wait complete. 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.877+0000 7f93f5074740 1 -- start start 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.877+0000 7f93f5074740 1 Processor -- start 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.878+0000 7f93f5074740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e10040 0x5627d4e2b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.878+0000 7f93f5074740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e115b0 0x5627d4e2bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.878+0000 7f93f5074740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e1d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.878+0000 7f93f5074740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1ae030 con 0x5627d4e10040 2024-06-03T01:24:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.878+0000 7f93f5074740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1ad930 con 0x5627d4e235d0 2024-06-03T01:24:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.878+0000 7f93f5074740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1aaa40 con 0x5627d4e115b0 2024-06-03T01:24:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.878+0000 7f93f5074740 10 client.0 _ll_register_callbacks cb 0x7f93dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-03T01:24:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.879+0000 7f93eaa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e115b0 0x5627d4e2bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.879+0000 7f93eaa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e115b0 0x5627d4e2bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:36102/0 (socket says 172.21.15.94:36102) 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.879+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e1d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.879+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e1d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:37272/0 (socket says 172.21.15.94:37272) 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.879+0000 7f93ebe00640 1 -- 172.21.15.94:0/2109685941 learned_addr learned my addr 172.21.15.94:0/2109685941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.879+0000 7f93eb400640 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e10040 0x5627d4e2b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93ebe00640 1 -- 172.21.15.94:0/2109685941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e115b0 msgr2=0x5627d4e2bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93ebe00640 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e115b0 0x5627d4e2bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93ebe00640 1 -- 172.21.15.94:0/2109685941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e10040 msgr2=0x5627d4e2b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-03T01:24:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93ebe00640 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e10040 0x5627d4e2b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93ebe00640 1 -- 172.21.15.94:0/2109685941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d4e1e080 con 0x5627d4e235d0 2024-06-03T01:24:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93eb400640 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e10040 0x5627d4e2b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:24:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93ebe00640 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e1d8a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93e40982a0 tx=0x7f93e409be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9e300f5854913c server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.880+0000 7f93d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-06-03T01:24:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.881+0000 7f93d4c00640 1 -- 172.21.15.94:0/2109685941 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40ba060 con 0x5627d4e235d0 2024-06-03T01:24:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.881+0000 7f93d4c00640 1 -- 172.21.15.94:0/2109685941 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9475+0+0 (secure 0 0 0) 0x7f93e4004b40 con 0x5627d4e235d0 2024-06-03T01:24:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.881+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e4009820 con 0x5627d4e235d0 2024-06-03T01:24:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.882+0000 7f93d4c00640 1 -- 172.21.15.94:0/2109685941 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f93e40a9050 con 0x5627d4e235d0 2024-06-03T01:24:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.882+0000 7f93f5074740 10 client.26390 fetch_fsmap learned FSMap version 68 2024-06-03T01:24:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.882+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5627d4e3b5a0 con 0x5627d4e235d0 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.882+0000 7f93d4c00640 1 -- 172.21.15.94:0/2109685941 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e4009820 con 0x5627d4e235d0 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.884+0000 7f93d4c00640 1 -- 172.21.15.94:0/2109685941 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 68) ==== 8398+0+0 (secure 0 0 0) 0x7f93e409b320 con 0x5627d4e235d0 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.884+0000 7f93f5074740 10 client.26390 fetch_fsmap finished waiting for FSMap version 68 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.884+0000 7f93f5074740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.884+0000 7f93f5074740 20 client.26390 populate_metadata read hostname 'smithi094' 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.884+0000 7f93f5074740 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e18e70 0x5627d4ea9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.884+0000 7f93f5074740 4 client.26390 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-03T01:24:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.884+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 --> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627d4d7bea0 con 0x5627d4e18e70 2024-06-03T01:24:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.885+0000 7f93eb400640 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e18e70 0x5627d4ea9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.885+0000 7f93eb400640 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e18e70 0x5627d4ea9450 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc1d62a5ea00a03 server_cookie=c1ad8bdacabe93d0 in_seq=0 out_seq=0 2024-06-03T01:24:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:03.885+0000 7f93d4c00640 10 client.26390 ms_handle_connect on v2:172.21.15.107:6838/1423733861 2024-06-03T01:24:04.507 INFO:tasks.workunit.client.0.smithi094.stdout: 10 4289 36.23 MB/sec warmup 11 sec latency 564.828 ms 2024-06-03T01:24:04.584 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:04 smithi107 ceph-mon[36622]: pgmap v428: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-06-03T01:24:04.601 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:04 smithi094 ceph-mon[32119]: pgmap v428: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-06-03T01:24:04.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:04 smithi179 ceph-mon[37041]: pgmap v428: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-06-03T01:24:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.735+0000 7f93d4c00640 1 -- 172.21.15.94:0/2109685941 <== mds.0 v2:172.21.15.107:6838/1423733861 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5627d4d7bea0 con 0x5627d4e18e70 2024-06-03T01:24:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.735+0000 7f93d4c00640 10 client.26390 handle_command_reply: tid=0 2024-06-03T01:24:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.736+0000 7f93f5074740 1 client.26390 shutdown 2024-06-03T01:24:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.736+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 msgr2=0x5627d4e1d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.736+0000 7f93f5074740 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e1d8a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93e40982a0 tx=0x7f93e409be40 comp rx=0 tx=0).stop 2024-06-03T01:24:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 shutdown_connections 2024-06-03T01:24:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93f5074740 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:6838/1423733861,v1:172.21.15.107:6839/1423733861] conn(0x5627d4e18e70 0x5627d4ea9450 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93f5074740 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5627d4e235d0 0x5627d4e1d8a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93f5074740 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5627d4e115b0 0x5627d4e2bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93d4c00640 0 client.26390 ms_handle_reset on v2:172.21.15.107:6838/1423733861 2024-06-03T01:24:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93f5074740 1 --2- 172.21.15.94:0/2109685941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5627d4e10040 0x5627d4e2b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 >> 172.21.15.94:0/2109685941 conn(0x5627d4dd8f80 msgr2=0x5627d4e1eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.737+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 shutdown_connections 2024-06-03T01:24:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.738+0000 7f93f5074740 1 -- 172.21.15.94:0/2109685941 wait complete. 2024-06-03T01:24:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.738+0000 7f93f5074740 20 client.26390 trim_cache size 0 max 16384 2024-06-03T01:24:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-06-03T01:24:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-06-03T01:24:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-06-03T01:24:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f93d00a2e90 msgr2=0x7f93d00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f93d00a2e90 0x7f93d00a5350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5627d4e21c10 tx=0x7f93d8012040 comp rx=0 tx=0).stop 2024-06-03T01:24:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec19ec70 msgr2=0x7f93ec19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec19ec70 0x7f93ec19f120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ec50 tx=0x7f93e400c6a0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 shutdown_connections 2024-06-03T01:24:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f93d00a2e90 0x7f93d00a5350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93ec19ec70 0x7f93ec19f120 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93ec108ec0 0x7f93ec19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 --2- 172.21.15.94:0/1611988883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec103df0 0x7f93ec19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.739+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 >> 172.21.15.94:0/1611988883 conn(0x7f93ec0ff690 msgr2=0x7f93ec100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.740+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 shutdown_connections 2024-06-03T01:24:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-06-03T01:24:04.740+0000 7f93f2200640 1 -- 172.21.15.94:0/1611988883 wait complete. 2024-06-03T01:24:05.054 INFO:tasks.workunit.client.0.smithi094.stdout:[2262] write failed on handle 10357 (Resource temporarily unavailable) 2024-06-03T01:24:05.054 INFO:tasks.workunit.client.0.smithi094.stdout:Child failed with status 1 2024-06-03T01:24:05.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:24:05.151 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-06-03T01:24:05.151 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-03T01:24:05.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:05 smithi094 ceph-mon[32119]: osdmap e116: 12 total, 12 up, 12 in 2024-06-03T01:24:05.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:05 smithi179 ceph-mon[37041]: osdmap e116: 12 total, 12 up, 12 in 2024-06-03T01:24:05.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:05 smithi107 ceph-mon[36622]: osdmap e116: 12 total, 12 up, 12 in 2024-06-03T01:24:06.285 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi094 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-06-03T01:24:06.524 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=82e80c0621444e838dfa91c94bbd77ec Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi094 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-06-03T01:24:06.526 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-03T01:24:06.536 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:24:06.560 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:06 smithi094 ceph-mon[32119]: pgmap v430: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 303 op/s 2024-06-03T01:24:06.563 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-06-03T01:24:06.577 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-06-03T01:24:06.577 INFO:teuthology.task.exec:Executing custom commands... 2024-06-03T01:24:06.577 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:24:06.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-06-03T01:24:06.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:06 smithi179 ceph-mon[37041]: pgmap v430: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 303 op/s 2024-06-03T01:24:06.698 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-06-03T01:24:06.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-03T01:24:06.784 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:06 smithi107 ceph-mon[36622]: pgmap v430: 129 pgs: 129 active+clean; 145 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 303 op/s 2024-06-03T01:24:06.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.993+0000 7fdeeb000640 1 -- 172.21.15.94:0/1956914113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 msgr2=0x7fdee411d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.993+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1956914113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee411d570 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fded0009900 tx=0x7fded002a480 comp rx=0 tx=0).stop 2024-06-03T01:24:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.994+0000 7fdeeb000640 1 -- 172.21.15.94:0/1956914113 shutdown_connections 2024-06-03T01:24:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.994+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1956914113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee411d570 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.994+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1956914113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee4110570 0x7fdee4108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.994+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1956914113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee4110190 0x7fdee41089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.994+0000 7fdeeb000640 1 -- 172.21.15.94:0/1956914113 >> 172.21.15.94:0/1956914113 conn(0x7fdee406ca70 msgr2=0x7fdee406ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.995+0000 7fdeeb000640 1 -- 172.21.15.94:0/1956914113 shutdown_connections 2024-06-03T01:24:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.995+0000 7fdeeb000640 1 -- 172.21.15.94:0/1956914113 wait complete. 2024-06-03T01:24:06.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.996+0000 7fdeeb000640 1 Processor -- start 2024-06-03T01:24:06.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.997+0000 7fdeeb000640 1 -- start start 2024-06-03T01:24:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.997+0000 7fdeeb000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee4110190 0x7fdee41a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.997+0000 7fdeeb000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee41a7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.997+0000 7fdeeb000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee41ab8d0 0x7fdee41abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.997+0000 7fdeeb000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4120bf0 con 0x7fdee4110190 2024-06-03T01:24:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.997+0000 7fdeeb000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4120a70 con 0x7fdee41ab8d0 2024-06-03T01:24:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.997+0000 7fdeeb000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4120d70 con 0x7fdee411b180 2024-06-03T01:24:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.998+0000 7fdee3400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee41a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:06.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.998+0000 7fdee9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee41ab8d0 0x7fdee41abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.998+0000 7fdee3400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee41a7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:57370/0 (socket says 172.21.15.94:57370) 2024-06-03T01:24:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.998+0000 7fdee3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee4110190 0x7fdee41a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.998+0000 7fdee9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee41ab8d0 0x7fdee41abd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:46630/0 (socket says 172.21.15.94:46630) 2024-06-03T01:24:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.998+0000 7fdee3400640 1 -- 172.21.15.94:0/1180502081 learned_addr learned my addr 172.21.15.94:0/1180502081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.999+0000 7fdee3400640 1 -- 172.21.15.94:0/1180502081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee41ab8d0 msgr2=0x7fdee41abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.999+0000 7fdee3400640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee41ab8d0 0x7fdee41abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.999+0000 7fdee3400640 1 -- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee4110190 msgr2=0x7fdee41a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.999+0000 7fdee3400640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee4110190 0x7fdee41a7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:06.999+0000 7fdee3400640 1 -- 172.21.15.94:0/1180502081 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee41ac3c0 con 0x7fdee411b180 2024-06-03T01:24:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.000+0000 7fdee3400640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee41a7550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fded400dbf0 tx=0x7fded400b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.000+0000 7fdee0c00640 1 -- 172.21.15.94:0/1180502081 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4017070 con 0x7fdee411b180 2024-06-03T01:24:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.001+0000 7fdee0c00640 1 -- 172.21.15.94:0/1180502081 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded4003e50 con 0x7fdee411b180 2024-06-03T01:24:07.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.001+0000 7fdee0c00640 1 -- 172.21.15.94:0/1180502081 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded40106c0 con 0x7fdee411b180 2024-06-03T01:24:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.001+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee41ac6b0 con 0x7fdee411b180 2024-06-03T01:24:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.002+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee41b3e50 con 0x7fdee411b180 2024-06-03T01:24:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.004+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee404f760 con 0x7fdee411b180 2024-06-03T01:24:07.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.006+0000 7fdee0c00640 1 -- 172.21.15.94:0/1180502081 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fded4003970 con 0x7fdee411b180 2024-06-03T01:24:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.008+0000 7fdee0c00640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdec80a2e20 0x7fdec80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.008+0000 7fdee0c00640 1 -- 172.21.15.94:0/1180502081 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9563+0+0 (secure 0 0 0) 0x7fded40cf4b0 con 0x7fdee411b180 2024-06-03T01:24:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.008+0000 7fdee3e00640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdec80a2e20 0x7fdec80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.009+0000 7fdee3e00640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdec80a2e20 0x7fdec80a52e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005fd0 tx=0x7fdedc005850 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.012+0000 7fdee0c00640 1 -- 172.21.15.94:0/1180502081 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fded4020030 con 0x7fdee411b180 2024-06-03T01:24:07.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.235+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 --> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fdee41107a0 con 0x7fdec80a2e20 2024-06-03T01:24:07.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.250+0000 7fdee0c00640 1 -- 172.21.15.94:0/1180502081 <== mgr.15084 v2:172.21.15.94:6800/2692604051 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fdedc00fb30 con 0x7fdec80a2e20 2024-06-03T01:24:07.251 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-03T00:00:00", "created": "2024-06-03T01:10:22", "first": "2024-06-03T01:11:00", "last": "2024-06-03T01:24:00", "last_pruned": "2024-06-03T01:24:00", "created_count": 14, "pruned_count": 8, "active": true} 2024-06-03T01:24:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.252+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdec80a2e20 msgr2=0x7fdec80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.252+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdec80a2e20 0x7fdec80a52e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005fd0 tx=0x7fdedc005850 comp rx=0 tx=0).stop 2024-06-03T01:24:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.252+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 msgr2=0x7fdee41a7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.252+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee41a7550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fded400dbf0 tx=0x7fded400b630 comp rx=0 tx=0).stop 2024-06-03T01:24:07.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 shutdown_connections 2024-06-03T01:24:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fdec80a2e20 0x7fdec80a52e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee41ab8d0 0x7fdee41abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdee411b180 0x7fdee41a7550 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 --2- 172.21.15.94:0/1180502081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdee4110190 0x7fdee41a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 >> 172.21.15.94:0/1180502081 conn(0x7fdee406ca70 msgr2=0x7fdee410de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 shutdown_connections 2024-06-03T01:24:07.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.253+0000 7fdeeb000640 1 -- 172.21.15.94:0/1180502081 wait complete. 2024-06-03T01:24:07.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-03T01:24:07.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 -- 172.21.15.94:0/3870792004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0300767f0 msgr2=0x7fc0301122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 --2- 172.21.15.94:0/3870792004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0300767f0 0x7fc0301122f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01802a410 comp rx=0 tx=0).stop 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 -- 172.21.15.94:0/3870792004 shutdown_connections 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 --2- 172.21.15.94:0/3870792004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc030112830 0x7fc030114c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 --2- 172.21.15.94:0/3870792004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0300767f0 0x7fc0301122f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 --2- 172.21.15.94:0/3870792004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc030075ed0 0x7fc0300762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 -- 172.21.15.94:0/3870792004 >> 172.21.15.94:0/3870792004 conn(0x7fc0300ff6a0 msgr2=0x7fc030101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 -- 172.21.15.94:0/3870792004 shutdown_connections 2024-06-03T01:24:07.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.476+0000 7fc035c00640 1 -- 172.21.15.94:0/3870792004 wait complete. 2024-06-03T01:24:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 Processor -- start 2024-06-03T01:24:07.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 -- start start 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc030075ed0 0x7fc0301a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0300767f0 0x7fc0301a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc030112830 0x7fc03019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0301183e0 con 0x7fc030075ed0 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc030118260 con 0x7fc0300767f0 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.477+0000 7fc035c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc030118560 con 0x7fc030112830 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc030075ed0 0x7fc0301a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:07.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc030075ed0 0x7fc0301a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57596/0 (socket says 172.21.15.94:57596) 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 -- 172.21.15.94:0/2787717751 learned_addr learned my addr 172.21.15.94:0/2787717751 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02e000640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0300767f0 0x7fc0301a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02f400640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc030112830 0x7fc03019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 -- 172.21.15.94:0/2787717751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc030112830 msgr2=0x7fc03019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc030112830 0x7fc03019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 -- 172.21.15.94:0/2787717751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0300767f0 msgr2=0x7fc0301a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0300767f0 0x7fc0301a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02ea00640 1 -- 172.21.15.94:0/2787717751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03019eda0 con 0x7fc030075ed0 2024-06-03T01:24:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.478+0000 7fc02f400640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc030112830 0x7fc03019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:24:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.479+0000 7fc02ea00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc030075ed0 0x7fc0301a4470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc020009d50 tx=0x7fc02000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.479+0000 7fc027400640 1 -- 172.21.15.94:0/2787717751 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020003b70 con 0x7fc030075ed0 2024-06-03T01:24:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.479+0000 7fc027400640 1 -- 172.21.15.94:0/2787717751 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc020003d10 con 0x7fc030075ed0 2024-06-03T01:24:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.479+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03019f090 con 0x7fc030075ed0 2024-06-03T01:24:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.479+0000 7fc027400640 1 -- 172.21.15.94:0/2787717751 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020010a20 con 0x7fc030075ed0 2024-06-03T01:24:07.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.479+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0301ab330 con 0x7fc030075ed0 2024-06-03T01:24:07.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.480+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03004f760 con 0x7fc030075ed0 2024-06-03T01:24:07.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.482+0000 7fc027400640 1 -- 172.21.15.94:0/2787717751 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc020003ec0 con 0x7fc030075ed0 2024-06-03T01:24:07.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.483+0000 7fc027400640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc0140a2da0 0x7fc0140a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.484+0000 7fc027400640 1 -- 172.21.15.94:0/2787717751 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9563+0+0 (secure 0 0 0) 0x7fc0200cff70 con 0x7fc030075ed0 2024-06-03T01:24:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.484+0000 7fc02e000640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc0140a2da0 0x7fc0140a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:07.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.484+0000 7fc02e000640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc0140a2da0 0x7fc0140a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc018005b50 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:07.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.489+0000 7fc027400640 1 -- 172.21.15.94:0/2787717751 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc02009c720 con 0x7fc030075ed0 2024-06-03T01:24:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.714+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 --> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fc030114980 con 0x7fc0140a2da0 2024-06-03T01:24:07.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.727+0000 7fc027400640 1 -- 172.21.15.94:0/2787717751 <== mgr.15084 v2:172.21.15.94:6800/2692604051 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fc030114980 con 0x7fc0140a2da0 2024-06-03T01:24:07.729 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-06-03T01:24:07.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc0140a2da0 msgr2=0x7fc0140a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc0140a2da0 0x7fc0140a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc018000c00 tx=0x7fc018005b50 comp rx=0 tx=0).stop 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc030075ed0 msgr2=0x7fc0301a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc030075ed0 0x7fc0301a4470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc020009d50 tx=0x7fc02000c6a0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 shutdown_connections 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fc0140a2da0 0x7fc0140a5260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc030112830 0x7fc03019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0300767f0 0x7fc0301a49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 --2- 172.21.15.94:0/2787717751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc030075ed0 0x7fc0301a4470 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 >> 172.21.15.94:0/2787717751 conn(0x7fc0300ff6a0 msgr2=0x7fc030114270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:07.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 shutdown_connections 2024-06-03T01:24:07.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:07.730+0000 7fc035c00640 1 -- 172.21.15.94:0/2787717751 wait complete. 2024-06-03T01:24:07.750 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-03T01:24:07.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-03T01:24:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.067+0000 7f864d400640 1 -- 172.21.15.94:0/1002590344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8648110100 msgr2=0x7f864810dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.067+0000 7f864d400640 1 --2- 172.21.15.94:0/1002590344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8648110100 0x7f864810dce0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8638009900 tx=0x7f863802a420 comp rx=0 tx=0).stop 2024-06-03T01:24:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.067+0000 7f864d400640 1 -- 172.21.15.94:0/1002590344 shutdown_connections 2024-06-03T01:24:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.067+0000 7f864d400640 1 --2- 172.21.15.94:0/1002590344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f864810e900 0x7f864811d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.067+0000 7f864d400640 1 --2- 172.21.15.94:0/1002590344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648110570 0x7f864810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.067+0000 7f864d400640 1 --2- 172.21.15.94:0/1002590344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8648110100 0x7f864810dce0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.067+0000 7f864d400640 1 -- 172.21.15.94:0/1002590344 >> 172.21.15.94:0/1002590344 conn(0x7f864806cac0 msgr2=0x7f864806ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:08.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.068+0000 7f864d400640 1 -- 172.21.15.94:0/1002590344 shutdown_connections 2024-06-03T01:24:08.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.068+0000 7f864d400640 1 -- 172.21.15.94:0/1002590344 wait complete. 2024-06-03T01:24:08.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.070+0000 7f864d400640 1 Processor -- start 2024-06-03T01:24:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.071+0000 7f864d400640 1 -- start start 2024-06-03T01:24:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.071+0000 7f864d400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8648110100 0x7f86481a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.072+0000 7f864d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86481a76c0 0x7f86481aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.072+0000 7f864d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86481abf80 0x7f86481ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.072+0000 7f864d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8648120c00 con 0x7f86481a76c0 2024-06-03T01:24:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.072+0000 7f864d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8648120a80 con 0x7f86481abf80 2024-06-03T01:24:08.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.072+0000 7f864d400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8648120d80 con 0x7f8648110100 2024-06-03T01:24:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.072+0000 7f8646800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8648110100 0x7f86481a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.072+0000 7f8647200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86481abf80 0x7f86481ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.073+0000 7f8646800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8648110100 0x7f86481a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:57410/0 (socket says 172.21.15.94:57410) 2024-06-03T01:24:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.073+0000 7f8646800640 1 -- 172.21.15.94:0/182278443 learned_addr learned my addr 172.21.15.94:0/182278443 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:08.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.073+0000 7f8645e00640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86481a76c0 0x7f86481aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.074+0000 7f8646800640 1 -- 172.21.15.94:0/182278443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86481abf80 msgr2=0x7f86481ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.074+0000 7f8646800640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86481abf80 0x7f86481ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.074+0000 7f8646800640 1 -- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86481a76c0 msgr2=0x7f86481aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.074+0000 7f8646800640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86481a76c0 0x7f86481aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.074+0000 7f8646800640 1 -- 172.21.15.94:0/182278443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481b3cf0 con 0x7f8648110100 2024-06-03T01:24:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.075+0000 7f8646800640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8648110100 0x7f86481a7180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f863802aa10 tx=0x7f8638003d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.075+0000 7f863f400640 1 -- 172.21.15.94:0/182278443 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863803f070 con 0x7f8648110100 2024-06-03T01:24:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.076+0000 7f863f400640 1 -- 172.21.15.94:0/182278443 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f863803b460 con 0x7f8648110100 2024-06-03T01:24:08.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.076+0000 7f863f400640 1 -- 172.21.15.94:0/182278443 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638035030 con 0x7f8648110100 2024-06-03T01:24:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.076+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481b3ec0 con 0x7f8648110100 2024-06-03T01:24:08.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.076+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481b4270 con 0x7f8648110100 2024-06-03T01:24:08.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.080+0000 7f863f400640 1 -- 172.21.15.94:0/182278443 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8638042070 con 0x7f8648110100 2024-06-03T01:24:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.081+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86480687c0 con 0x7f8648110100 2024-06-03T01:24:08.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.082+0000 7f863f400640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f862c0a2d80 0x7f862c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.082+0000 7f863f400640 1 -- 172.21.15.94:0/182278443 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9563+0+0 (secure 0 0 0) 0x7f86380efe80 con 0x7f8648110100 2024-06-03T01:24:08.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.085+0000 7f8645e00640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f862c0a2d80 0x7f862c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.089+0000 7f8645e00640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f862c0a2d80 0x7f862c0a5240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86340058c0 tx=0x7f8634005c60 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:08.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.091+0000 7f863f400640 1 -- 172.21.15.94:0/182278443 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f86380bc630 con 0x7f8648110100 2024-06-03T01:24:08.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.476+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f864810e960 con 0x7f8648110100 2024-06-03T01:24:08.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.478+0000 7f863f400640 1 -- 172.21.15.94:0/182278443 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+25424 (secure 0 0 0) 0x7f86380bbf30 con 0x7f8648110100 2024-06-03T01:24:08.482 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:24:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":116,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:24:04.305387+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/549573361":"2024-06-04T01:10:20.680471+0000","172.21.15.94:0/2999518385":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6800/4120992823":"2024-06-04T01:10:20.680471+0000","172.21.15.94:0/3717514300":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/1904227986":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6801/4120992823":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:24:08.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f862c0a2d80 msgr2=0x7f862c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f862c0a2d80 0x7f862c0a5240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f86340058c0 tx=0x7f8634005c60 comp rx=0 tx=0).stop 2024-06-03T01:24:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8648110100 msgr2=0x7f86481a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8648110100 0x7f86481a7180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f863802aa10 tx=0x7f8638003d50 comp rx=0 tx=0).stop 2024-06-03T01:24:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 shutdown_connections 2024-06-03T01:24:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f862c0a2d80 0x7f862c0a5240 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86481abf80 0x7f86481ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86481a76c0 0x7f86481aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.486+0000 7f864d400640 1 --2- 172.21.15.94:0/182278443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8648110100 0x7f86481a7180 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.487+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 >> 172.21.15.94:0/182278443 conn(0x7f864806cac0 msgr2=0x7f8648119c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.487+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 shutdown_connections 2024-06-03T01:24:08.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.487+0000 7f864d400640 1 -- 172.21.15.94:0/182278443 wait complete. 2024-06-03T01:24:08.519 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:08 smithi094 ceph-mon[32119]: pgmap v431: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 800 op/s 2024-06-03T01:24:08.519 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:08 smithi094 ceph-mon[32119]: from='client.26416 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:08.520 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-03T00:59:20.779028+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-03T01:04:48.698832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-03T01:04:53.283627+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-03T01:05:06.876021+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-03T01:24:08.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-03T01:24:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:08 smithi179 ceph-mon[37041]: pgmap v431: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 800 op/s 2024-06-03T01:24:08.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:08 smithi179 ceph-mon[37041]: from='client.26416 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:08 smithi107 ceph-mon[36622]: pgmap v431: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 800 op/s 2024-06-03T01:24:08.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:08 smithi107 ceph-mon[36622]: from='client.26416 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.871+0000 7f1817200640 1 -- 172.21.15.94:0/4280224618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1810075ed0 msgr2=0x7f18100762b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.871+0000 7f1817200640 1 --2- 172.21.15.94:0/4280224618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1810075ed0 0x7f18100762b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1800009930 tx=0x7f180002a420 comp rx=0 tx=0).stop 2024-06-03T01:24:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.872+0000 7f1817200640 1 -- 172.21.15.94:0/4280224618 shutdown_connections 2024-06-03T01:24:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.872+0000 7f1817200640 1 --2- 172.21.15.94:0/4280224618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1810112880 0x7f1810114c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.872+0000 7f1817200640 1 --2- 172.21.15.94:0/4280224618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18100767f0 0x7f1810112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.872+0000 7f1817200640 1 --2- 172.21.15.94:0/4280224618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1810075ed0 0x7f18100762b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.872+0000 7f1817200640 1 -- 172.21.15.94:0/4280224618 >> 172.21.15.94:0/4280224618 conn(0x7f18100ff730 msgr2=0x7f1810101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.873+0000 7f1817200640 1 -- 172.21.15.94:0/4280224618 shutdown_connections 2024-06-03T01:24:08.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.873+0000 7f1817200640 1 -- 172.21.15.94:0/4280224618 wait complete. 2024-06-03T01:24:08.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.875+0000 7f1817200640 1 Processor -- start 2024-06-03T01:24:08.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.875+0000 7f1817200640 1 -- start start 2024-06-03T01:24:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.876+0000 7f1817200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1810075ed0 0x7f18101a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.876+0000 7f1817200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18100767f0 0x7f18101a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.877+0000 7f1817200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1810112880 0x7f181019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.877+0000 7f1817200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f18101184b0 con 0x7f1810112880 2024-06-03T01:24:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.877+0000 7f1817200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1810118330 con 0x7f18100767f0 2024-06-03T01:24:08.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.877+0000 7f1817200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1810118630 con 0x7f1810075ed0 2024-06-03T01:24:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.877+0000 7f180fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1810075ed0 0x7f18101a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.877+0000 7f180f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18100767f0 0x7f18101a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.878+0000 7f180f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18100767f0 0x7f18101a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:46686/0 (socket says 172.21.15.94:46686) 2024-06-03T01:24:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.878+0000 7f180f400640 1 -- 172.21.15.94:0/3186305577 learned_addr learned my addr 172.21.15.94:0/3186305577 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.878+0000 7f1815200640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1810112880 0x7f181019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.879+0000 7f180f400640 1 -- 172.21.15.94:0/3186305577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1810075ed0 msgr2=0x7f18101a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.879+0000 7f180f400640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1810075ed0 0x7f18101a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.879+0000 7f180f400640 1 -- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1810112880 msgr2=0x7f181019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:08.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.879+0000 7f180f400640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1810112880 0x7f181019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:08.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.879+0000 7f180f400640 1 -- 172.21.15.94:0/3186305577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181019f020 con 0x7f18100767f0 2024-06-03T01:24:08.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.879+0000 7f180f400640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18100767f0 0x7f18101a4bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b4a0 tx=0x7f17fc00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:08.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.880+0000 7f180cc00640 1 -- 172.21.15.94:0/3186305577 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc014070 con 0x7f18100767f0 2024-06-03T01:24:08.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.880+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181019f310 con 0x7f18100767f0 2024-06-03T01:24:08.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.880+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18101ab4e0 con 0x7f18100767f0 2024-06-03T01:24:08.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.881+0000 7f180cc00640 1 -- 172.21.15.94:0/3186305577 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17fc010af0 con 0x7f18100767f0 2024-06-03T01:24:08.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.881+0000 7f180cc00640 1 -- 172.21.15.94:0/3186305577 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc014070 con 0x7f18100767f0 2024-06-03T01:24:08.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.883+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1810077610 con 0x7f18100767f0 2024-06-03T01:24:08.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.889+0000 7f180cc00640 1 -- 172.21.15.94:0/3186305577 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f17fc017070 con 0x7f18100767f0 2024-06-03T01:24:08.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.891+0000 7f180cc00640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f17f40a3000 0x7f17f40a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:08.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.891+0000 7f180cc00640 1 -- 172.21.15.94:0/3186305577 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7f17fc0ce740 con 0x7f18100767f0 2024-06-03T01:24:08.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.892+0000 7f180fe00640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f17f40a3000 0x7f17f40a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:08.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.894+0000 7f180fe00640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f17f40a3000 0x7f17f40a54c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800000bc0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:08.896+0000 7f180cc00640 1 -- 172.21.15.94:0/3186305577 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f17fc09b050 con 0x7f18100767f0 2024-06-03T01:24:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.195+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f181010da10 con 0x7f18100767f0 2024-06-03T01:24:09.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.195+0000 7f180cc00640 1 -- 172.21.15.94:0/3186305577 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) ==== 93+0+10 (secure 0 0 0) 0x7f17fc09a950 con 0x7f18100767f0 2024-06-03T01:24:09.196 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-06-03T01:24:09.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f17f40a3000 msgr2=0x7f17f40a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f17f40a3000 0x7f17f40a54c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800000bc0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18100767f0 msgr2=0x7f18101a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18100767f0 0x7f18101a4bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b4a0 tx=0x7f17fc00b960 comp rx=0 tx=0).stop 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 shutdown_connections 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f17f40a3000 0x7f17f40a54c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1810112880 0x7f181019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18100767f0 0x7f18101a4bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 --2- 172.21.15.94:0/3186305577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1810075ed0 0x7f18101a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 >> 172.21.15.94:0/3186305577 conn(0x7f18100ff730 msgr2=0x7f1810114010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 shutdown_connections 2024-06-03T01:24:09.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.198+0000 7f1817200640 1 -- 172.21.15.94:0/3186305577 wait complete. 2024-06-03T01:24:09.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-03T01:24:09.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.401+0000 7fd4e4e00640 1 -- 172.21.15.94:0/933302004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e0074690 msgr2=0x7fd4e010db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.401+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/933302004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e0074690 0x7fd4e010db90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009900 tx=0x7fd4cc02a480 comp rx=0 tx=0).stop 2024-06-03T01:24:09.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.401+0000 7fd4e4e00640 1 -- 172.21.15.94:0/933302004 shutdown_connections 2024-06-03T01:24:09.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.401+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/933302004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e010e0d0 0x7fd4e01104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.401+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/933302004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e0074690 0x7fd4e010db90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.401+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/933302004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0073d70 0x7fd4e0074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.401+0000 7fd4e4e00640 1 -- 172.21.15.94:0/933302004 >> 172.21.15.94:0/933302004 conn(0x7fd4e00fd340 msgr2=0x7fd4e00ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:09.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.402+0000 7fd4e4e00640 1 -- 172.21.15.94:0/933302004 shutdown_connections 2024-06-03T01:24:09.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.402+0000 7fd4e4e00640 1 -- 172.21.15.94:0/933302004 wait complete. 2024-06-03T01:24:09.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.402+0000 7fd4e4e00640 1 Processor -- start 2024-06-03T01:24:09.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4e4e00640 1 -- start start 2024-06-03T01:24:09.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4e4e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e0073d70 0x7fd4e019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4e4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0074690 0x7fd4e019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4e4e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e010e0d0 0x7fd4e019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4e4e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0113ea0 con 0x7fd4e010e0d0 2024-06-03T01:24:09.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4e4e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0113d20 con 0x7fd4e0074690 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4e4e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0114020 con 0x7fd4e0073d70 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e010e0d0 0x7fd4e019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4dfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e010e0d0 0x7fd4e019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57634/0 (socket says 172.21.15.94:57634) 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4dfe00640 1 -- 172.21.15.94:0/4044957216 learned_addr learned my addr 172.21.15.94:0/4044957216 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4dea00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0074690 0x7fd4e019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.403+0000 7fd4df400640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e0073d70 0x7fd4e019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4dfe00640 1 -- 172.21.15.94:0/4044957216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e0073d70 msgr2=0x7fd4e019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4dfe00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e0073d70 0x7fd4e019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4dfe00640 1 -- 172.21.15.94:0/4044957216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0074690 msgr2=0x7fd4e019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4dfe00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0074690 0x7fd4e019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4dfe00640 1 -- 172.21.15.94:0/4044957216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e019f410 con 0x7fd4e010e0d0 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4df400640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e0073d70 0x7fd4e019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4dfe00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e010e0d0 0x7fd4e019ece0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800eb10 tx=0x7fd4d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4d7e00640 1 -- 172.21.15.94:0/4044957216 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8004b20 con 0x7fd4e010e0d0 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4d7e00640 1 -- 172.21.15.94:0/4044957216 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d8004cc0 con 0x7fd4e010e0d0 2024-06-03T01:24:09.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4d7e00640 1 -- 172.21.15.94:0/4044957216 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8010790 con 0x7fd4e010e0d0 2024-06-03T01:24:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e019f700 con 0x7fd4e010e0d0 2024-06-03T01:24:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.404+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e01a6f90 con 0x7fd4e010e0d0 2024-06-03T01:24:09.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.405+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e0067e50 con 0x7fd4e010e0d0 2024-06-03T01:24:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.407+0000 7fd4d7e00640 1 -- 172.21.15.94:0/4044957216 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd4d8005020 con 0x7fd4e010e0d0 2024-06-03T01:24:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.409+0000 7fd4d7e00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd4c40a2d00 0x7fd4c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.410+0000 7fd4d7e00640 1 -- 172.21.15.94:0/4044957216 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7fd4d80cff20 con 0x7fd4e010e0d0 2024-06-03T01:24:09.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.410+0000 7fd4df400640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd4c40a2d00 0x7fd4c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.410+0000 7fd4df400640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd4c40a2d00 0x7fd4c40a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009fc0 tx=0x7fd4d0005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:09.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.414+0000 7fd4d7e00640 1 -- 172.21.15.94:0/4044957216 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd4d809c720 con 0x7fd4e010e0d0 2024-06-03T01:24:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:09 smithi094 ceph-mon[32119]: from='client.16281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:09.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:09 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/182278443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:24:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:09 smithi094 ceph-mon[32119]: osdmap e117: 12 total, 12 up, 12 in 2024-06-03T01:24:09.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:09 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3186305577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:24:09.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.601+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4e01092b0 con 0x7fd4e010e0d0 2024-06-03T01:24:09.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.602+0000 7fd4d7e00640 1 -- 172.21.15.94:0/4044957216 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) ==== 104+0+11 (secure 0 0 0) 0x7fd4d809c020 con 0x7fd4e010e0d0 2024-06-03T01:24:09.603 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-06-03T01:24:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.604+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd4c40a2d00 msgr2=0x7fd4c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.604+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd4c40a2d00 0x7fd4c40a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009fc0 tx=0x7fd4d0005f90 comp rx=0 tx=0).stop 2024-06-03T01:24:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.604+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e010e0d0 msgr2=0x7fd4e019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e010e0d0 0x7fd4e019ece0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800eb10 tx=0x7fd4d800c2d0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 shutdown_connections 2024-06-03T01:24:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fd4c40a2d00 0x7fd4c40a51c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e010e0d0 0x7fd4e019ece0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0074690 0x7fd4e019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 --2- 172.21.15.94:0/4044957216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e0073d70 0x7fd4e019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 >> 172.21.15.94:0/4044957216 conn(0x7fd4e00fd340 msgr2=0x7fd4e00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 shutdown_connections 2024-06-03T01:24:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.605+0000 7fd4e4e00640 1 -- 172.21.15.94:0/4044957216 wait complete. 2024-06-03T01:24:09.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:09 smithi179 ceph-mon[37041]: from='client.16281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:09.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:09 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/182278443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:24:09.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:09 smithi179 ceph-mon[37041]: osdmap e117: 12 total, 12 up, 12 in 2024-06-03T01:24:09.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:09 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3186305577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:24:09.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-03T01:24:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:09 smithi107 ceph-mon[36622]: from='client.16281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-03T01:24:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:09 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/182278443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:24:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:09 smithi107 ceph-mon[36622]: osdmap e117: 12 total, 12 up, 12 in 2024-06-03T01:24:09.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:09 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3186305577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-03T01:24:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.823+0000 7f3d0b200640 1 -- 172.21.15.94:0/1071987234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108bd0 msgr2=0x7f3d0410cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1071987234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108bd0 0x7f3d0410cbc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009900 tx=0x7f3cf002a400 comp rx=0 tx=0).stop 2024-06-03T01:24:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 -- 172.21.15.94:0/1071987234 shutdown_connections 2024-06-03T01:24:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1071987234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 0x7f3d04114e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1071987234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108bd0 0x7f3d0410cbc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1071987234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d04108220 0x7f3d04108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 -- 172.21.15.94:0/1071987234 >> 172.21.15.94:0/1071987234 conn(0x7f3d04075ee0 msgr2=0x7f3d04078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 -- 172.21.15.94:0/1071987234 shutdown_connections 2024-06-03T01:24:09.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.824+0000 7f3d0b200640 1 -- 172.21.15.94:0/1071987234 wait complete. 2024-06-03T01:24:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 Processor -- start 2024-06-03T01:24:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 -- start start 2024-06-03T01:24:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108220 0x7f3d041a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 0x7f3d041a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d0419e6e0 0x7f3d0419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04118380 con 0x7f3d0410d2a0 2024-06-03T01:24:09.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04118200 con 0x7f3d04108220 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d0b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04118500 con 0x7f3d0419e6e0 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.825+0000 7f3d03400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 0x7f3d041a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 0x7f3d041a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57652/0 (socket says 172.21.15.94:57652) 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 -- 172.21.15.94:0/1770217660 learned_addr learned my addr 172.21.15.94:0/1770217660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03e00640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108220 0x7f3d041a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d09200640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d0419e6e0 0x7f3d0419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 -- 172.21.15.94:0/1770217660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d0419e6e0 msgr2=0x7f3d0419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d0419e6e0 0x7f3d0419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 -- 172.21.15.94:0/1770217660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108220 msgr2=0x7f3d041a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108220 0x7f3d041a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 -- 172.21.15.94:0/1770217660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0419f2e0 con 0x7f3d0410d2a0 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.826+0000 7f3d03400640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 0x7f3d041a4b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3cf002a920 tx=0x7f3cf0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.827+0000 7f3d00c00640 1 -- 172.21.15.94:0/1770217660 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf003f070 con 0x7f3d0410d2a0 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.827+0000 7f3d00c00640 1 -- 172.21.15.94:0/1770217660 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf0031c90 con 0x7f3d0410d2a0 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.827+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0419f570 con 0x7f3d0410d2a0 2024-06-03T01:24:09.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.827+0000 7f3d00c00640 1 -- 172.21.15.94:0/1770217660 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0035030 con 0x7f3d0410d2a0 2024-06-03T01:24:09.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.827+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d041ab6b0 con 0x7f3d0410d2a0 2024-06-03T01:24:09.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.828+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d041096a0 con 0x7f3d0410d2a0 2024-06-03T01:24:09.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.831+0000 7f3d00c00640 1 -- 172.21.15.94:0/1770217660 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3cf0042070 con 0x7f3d0410d2a0 2024-06-03T01:24:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.832+0000 7f3d00c00640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3ce80a2d00 0x7f3ce80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.832+0000 7f3d00c00640 1 -- 172.21.15.94:0/1770217660 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7f3cf00efe70 con 0x7f3d0410d2a0 2024-06-03T01:24:09.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.832+0000 7f3d03e00640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3ce80a2d00 0x7f3ce80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:09.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.833+0000 7f3d03e00640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3ce80a2d00 0x7f3ce80a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009bd0 tx=0x7f3cf4005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:09.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:09.836+0000 7f3d00c00640 1 -- 172.21.15.94:0/1770217660 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3cf00bc020 con 0x7f3d0410d2a0 2024-06-03T01:24:10.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.027+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d04108bd0 con 0x7f3d0410d2a0 2024-06-03T01:24:10.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.028+0000 7f3d00c00640 1 -- 172.21.15.94:0/1770217660 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) ==== 100+0+11 (secure 0 0 0) 0x7f3cf00c0320 con 0x7f3d0410d2a0 2024-06-03T01:24:10.028 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-06-03T01:24:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3ce80a2d00 msgr2=0x7f3ce80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3ce80a2d00 0x7f3ce80a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009bd0 tx=0x7f3cf4005f90 comp rx=0 tx=0).stop 2024-06-03T01:24:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 msgr2=0x7f3d041a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 0x7f3d041a4b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3cf002a920 tx=0x7f3cf0003b20 comp rx=0 tx=0).stop 2024-06-03T01:24:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 shutdown_connections 2024-06-03T01:24:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f3ce80a2d00 0x7f3ce80a51c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d0419e6e0 0x7f3d0419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d0410d2a0 0x7f3d041a4b60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 --2- 172.21.15.94:0/1770217660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d04108220 0x7f3d041a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.030+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 >> 172.21.15.94:0/1770217660 conn(0x7f3d04075ee0 msgr2=0x7f3d040772a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.031+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 shutdown_connections 2024-06-03T01:24:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.031+0000 7f3d0b200640 1 -- 172.21.15.94:0/1770217660 wait complete. 2024-06-03T01:24:10.049 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-03T01:24:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 -- 172.21.15.94:0/2378033871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 msgr2=0x7f6a10074150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 --2- 172.21.15.94:0/2378033871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a10074150 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009900 tx=0x7f6a0402a480 comp rx=0 tx=0).stop 2024-06-03T01:24:10.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 -- 172.21.15.94:0/2378033871 shutdown_connections 2024-06-03T01:24:10.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 --2- 172.21.15.94:0/2378033871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1010e0c0 0x7f6a101104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 --2- 172.21.15.94:0/2378033871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10074690 0x7f6a1010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 --2- 172.21.15.94:0/2378033871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a10074150 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 -- 172.21.15.94:0/2378033871 >> 172.21.15.94:0/2378033871 conn(0x7f6a100fd370 msgr2=0x7f6a100ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:10.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 -- 172.21.15.94:0/2378033871 shutdown_connections 2024-06-03T01:24:10.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.233+0000 7f6a16e00640 1 -- 172.21.15.94:0/2378033871 wait complete. 2024-06-03T01:24:10.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.234+0000 7f6a16e00640 1 Processor -- start 2024-06-03T01:24:10.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.234+0000 7f6a16e00640 1 -- start start 2024-06-03T01:24:10.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a16e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a1019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a16e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10074690 0x7f6a1019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a16e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1010e0c0 0x7f6a1019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a16e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10113e40 con 0x7f6a10073d70 2024-06-03T01:24:10.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a16e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10113cc0 con 0x7f6a10074690 2024-06-03T01:24:10.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a16e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10113fc0 con 0x7f6a1010e0c0 2024-06-03T01:24:10.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a15a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a1019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a15a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a1019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57662/0 (socket says 172.21.15.94:57662) 2024-06-03T01:24:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a15a00640 1 -- 172.21.15.94:0/44755956 learned_addr learned my addr 172.21.15.94:0/44755956 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a15000640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10074690 0x7f6a1019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.235+0000 7f6a16400640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1010e0c0 0x7f6a1019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a15000640 1 -- 172.21.15.94:0/44755956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1010e0c0 msgr2=0x7f6a1019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a15000640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1010e0c0 0x7f6a1019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a15000640 1 -- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 msgr2=0x7f6a1019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a15000640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a1019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a15000640 1 -- 172.21.15.94:0/44755956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1019f360 con 0x7f6a10074690 2024-06-03T01:24:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a15a00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a1019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:24:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a15000640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10074690 0x7f6a1019a900 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005fd0 tx=0x7f69fc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a0e000640 1 -- 172.21.15.94:0/44755956 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc009c30 con 0x7f6a10074690 2024-06-03T01:24:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1019f650 con 0x7f6a10074690 2024-06-03T01:24:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a0e000640 1 -- 172.21.15.94:0/44755956 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69fc009dd0 con 0x7f6a10074690 2024-06-03T01:24:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a0e000640 1 -- 172.21.15.94:0/44755956 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc010460 con 0x7f6a10074690 2024-06-03T01:24:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.236+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a101a6ee0 con 0x7f6a10074690 2024-06-03T01:24:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.237+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a10068b30 con 0x7f6a10074690 2024-06-03T01:24:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.240+0000 7f6a0e000640 1 -- 172.21.15.94:0/44755956 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f69fc003ef0 con 0x7f6a10074690 2024-06-03T01:24:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.241+0000 7f6a0e000640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f69f80a2d00 0x7f69f80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.241+0000 7f6a0e000640 1 -- 172.21.15.94:0/44755956 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7f69fc0cf320 con 0x7f6a10074690 2024-06-03T01:24:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.241+0000 7f6a15a00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f69f80a2d00 0x7f69f80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.242+0000 7f6a15a00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f69f80a2d00 0x7f69f80a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a040096c0 tx=0x7f6a04005a20 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:10.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.243+0000 7f6a0e000640 1 -- 172.21.15.94:0/44755956 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f69fc017020 con 0x7f6a10074690 2024-06-03T01:24:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.447+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6a10109270 con 0x7f6a10074690 2024-06-03T01:24:10.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.449+0000 7f6a0e000640 1 -- 172.21.15.94:0/44755956 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v117) ==== 103+0+11 (secure 0 0 0) 0x7f69fc09b4d0 con 0x7f6a10074690 2024-06-03T01:24:10.449 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-06-03T01:24:10.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f69f80a2d00 msgr2=0x7f69f80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f69f80a2d00 0x7f69f80a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a040096c0 tx=0x7f6a04005a20 comp rx=0 tx=0).stop 2024-06-03T01:24:10.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10074690 msgr2=0x7f6a1019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10074690 0x7f6a1019a900 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005fd0 tx=0x7f69fc00dde0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 shutdown_connections 2024-06-03T01:24:10.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f69f80a2d00 0x7f69f80a51c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a1010e0c0 0x7f6a1019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10074690 0x7f6a1019a900 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 --2- 172.21.15.94:0/44755956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a10073d70 0x7f6a1019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 >> 172.21.15.94:0/44755956 conn(0x7f6a100fd370 msgr2=0x7f6a100feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:10.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 shutdown_connections 2024-06-03T01:24:10.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.451+0000 7f6a16e00640 1 -- 172.21.15.94:0/44755956 wait complete. 2024-06-03T01:24:10.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-03T01:24:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:10 smithi094 ceph-mon[32119]: pgmap v433: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 1k op/s 2024-06-03T01:24:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:10 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/4044957216' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T01:24:10.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:10 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1770217660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T01:24:10.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:10 smithi179 ceph-mon[37041]: pgmap v433: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 1k op/s 2024-06-03T01:24:10.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:10 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/4044957216' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T01:24:10.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:10 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1770217660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T01:24:10.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 -- 172.21.15.94:0/1598250386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 msgr2=0x7f0ba8105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 --2- 172.21.15.94:0/1598250386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba8105800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009900 tx=0x7f0b9002a510 comp rx=0 tx=0).stop 2024-06-03T01:24:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 -- 172.21.15.94:0/1598250386 shutdown_connections 2024-06-03T01:24:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 --2- 172.21.15.94:0/1598250386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ba8105d40 0x7f0ba81061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 --2- 172.21.15.94:0/1598250386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba8105800 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 --2- 172.21.15.94:0/1598250386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ba8100a20 0x7f0ba8100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 -- 172.21.15.94:0/1598250386 >> 172.21.15.94:0/1598250386 conn(0x7f0ba80779a0 msgr2=0x7f0ba8077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:10.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 -- 172.21.15.94:0/1598250386 shutdown_connections 2024-06-03T01:24:10.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.671+0000 7f0bae800640 1 -- 172.21.15.94:0/1598250386 wait complete. 2024-06-03T01:24:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.672+0000 7f0bae800640 1 Processor -- start 2024-06-03T01:24:10.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.672+0000 7f0bae800640 1 -- start start 2024-06-03T01:24:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0bae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ba8100a20 0x7f0ba81a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0bae800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba81a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0bae800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ba8105d40 0x7f0ba819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0bae800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba8118360 con 0x7f0ba8101340 2024-06-03T01:24:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0bae800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81181e0 con 0x7f0ba8100a20 2024-06-03T01:24:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0bae800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba81184e0 con 0x7f0ba8105d40 2024-06-03T01:24:10.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0b9ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba81a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0b9ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba81a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57674/0 (socket says 172.21.15.94:57674) 2024-06-03T01:24:10.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0b9ea00640 1 -- 172.21.15.94:0/704877772 learned_addr learned my addr 172.21.15.94:0/704877772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:10.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0ba7400640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ba8100a20 0x7f0ba81a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0ba7e00640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ba8105d40 0x7f0ba819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.673+0000 7f0b9ea00640 1 -- 172.21.15.94:0/704877772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ba8105d40 msgr2=0x7f0ba819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0b9ea00640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ba8105d40 0x7f0ba819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0b9ea00640 1 -- 172.21.15.94:0/704877772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ba8100a20 msgr2=0x7f0ba81a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:10.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0b9ea00640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ba8100a20 0x7f0ba81a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:10.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0b9ea00640 1 -- 172.21.15.94:0/704877772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba819ef80 con 0x7f0ba8101340 2024-06-03T01:24:10.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0ba7400640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ba8100a20 0x7f0ba81a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:24:10.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0b9ea00640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba81a4b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006010 tx=0x7f0b90004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:10.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0ba4c00640 1 -- 172.21.15.94:0/704877772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9003f030 con 0x7f0ba8101340 2024-06-03T01:24:10.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0ba4c00640 1 -- 172.21.15.94:0/704877772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b90004440 con 0x7f0ba8101340 2024-06-03T01:24:10.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba819f210 con 0x7f0ba8101340 2024-06-03T01:24:10.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0ba4c00640 1 -- 172.21.15.94:0/704877772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90035070 con 0x7f0ba8101340 2024-06-03T01:24:10.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.674+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba81ab3a0 con 0x7f0ba8101340 2024-06-03T01:24:10.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.675+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba8069330 con 0x7f0ba8101340 2024-06-03T01:24:10.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.677+0000 7f0ba4c00640 1 -- 172.21.15.94:0/704877772 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0b90042030 con 0x7f0ba8101340 2024-06-03T01:24:10.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.679+0000 7f0ba4c00640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0b8c0a2d00 0x7f0b8c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:10.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.679+0000 7f0ba4c00640 1 -- 172.21.15.94:0/704877772 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7f0b900ef490 con 0x7f0ba8101340 2024-06-03T01:24:10.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.679+0000 7f0ba7400640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0b8c0a2d00 0x7f0b8c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:10.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.680+0000 7f0ba7400640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0b8c0a2d00 0x7f0b8c0a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fc0 tx=0x7f0b98005f90 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:10.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.683+0000 7f0ba4c00640 1 -- 172.21.15.94:0/704877772 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b900bb640 con 0x7f0ba8101340 2024-06-03T01:24:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:10 smithi107 ceph-mon[36622]: pgmap v433: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.9 MiB/s wr, 1k op/s 2024-06-03T01:24:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:10 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/4044957216' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-03T01:24:10.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:10 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1770217660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-03T01:24:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.939+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f0ba81ab570 con 0x7f0ba8101340 2024-06-03T01:24:10.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.970+0000 7f0ba4c00640 1 -- 172.21.15.94:0/704877772 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740901 (secure 0 0 0) 0x7f0b900b7070 con 0x7f0ba8101340 2024-06-03T01:24:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:24:10.983 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:24:10.983 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 26, 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "active_gid": 15084, 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "active_name": "x", 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addrs": { 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-06-03T01:24:10.984 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2692604051 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2692604051 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:10.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addr": "172.21.15.94:6801/2692604051", 2024-06-03T01:24:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "active_change": "2024-06-03T01:10:20.680980+0000", 2024-06-03T01:24:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "active_mgr_features": 4540701547738038271, 2024-06-03T01:24:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-06-03T01:24:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "standbys": [ 2024-06-03T01:24:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:10.986 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 25202, 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "y", 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:10.987 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:10.988 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:10.989 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:10.990 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:10.991 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:10.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:10.993 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:10.994 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:10.995 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:10.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:10.999 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-06-03T01:24:11.000 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.001 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-06-03T01:24:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-06-03T01:24:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.002 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-06-03T01:24:11.003 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.004 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.005 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-06-03T01:24:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-06-03T01:24:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.008 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.009 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.010 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.011 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.015 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.016 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.017 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-06-03T01:24:11.018 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.019 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.020 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.021 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T01:24:11.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-06-03T01:24:11.023 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-06-03T01:24:11.024 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T01:24:11.025 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.026 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.027 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.029 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.030 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.031 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.032 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.033 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.035 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.036 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.038 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.039 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.040 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-06-03T01:24:11.041 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-06-03T01:24:11.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.042 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:11.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:11.042 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.042 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-06-03T01:24:11.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.045 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-06-03T01:24:11.046 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-06-03T01:24:11.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.048 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.050 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-06-03T01:24:11.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.050 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T01:24:11.050 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.051 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.052 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.053 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.054 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.056 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T01:24:11.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.059 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.060 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.061 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.061 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T01:24:11.061 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.061 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.061 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.062 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-06-03T01:24:11.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.064 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T01:24:11.065 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.066 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.067 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-06-03T01:24:11.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.073 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-06-03T01:24:11.074 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-06-03T01:24:11.075 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.078 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-06-03T01:24:11.079 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.080 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.081 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.082 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T01:24:11.083 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-06-03T01:24:11.084 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-06-03T01:24:11.085 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.088 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-06-03T01:24:11.089 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-06-03T01:24:11.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.090 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.090 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.090 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.090 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.091 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T01:24:11.091 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.091 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.091 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-06-03T01:24:11.091 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.092 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.093 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.094 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-06-03T01:24:11.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.094 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T01:24:11.094 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.095 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.098 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-06-03T01:24:11.098 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.098 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.098 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-06-03T01:24:11.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.099 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T01:24:11.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.100 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-06-03T01:24:11.100 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-06-03T01:24:11.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-06-03T01:24:11.101 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.103 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.104 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-06-03T01:24:11.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T01:24:11.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T01:24:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-06-03T01:24:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-06-03T01:24:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-06-03T01:24:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T01:24:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-06-03T01:24:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-06-03T01:24:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-06-03T01:24:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T01:24:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-06-03T01:24:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T01:24:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-06-03T01:24:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T01:24:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T01:24:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T01:24:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T01:24:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T01:24:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T01:24:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.198 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.200 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T01:24:11.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.206 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.206 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.206 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.207 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.207 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T01:24:11.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T01:24:11.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.208 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T01:24:11.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.212 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.214 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T01:24:11.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.216 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.218 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T01:24:11.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.221 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.222 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T01:24:11.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T01:24:11.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.222 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.223 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.224 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.225 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.226 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T01:24:11.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.229 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.229 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T01:24:11.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T01:24:11.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.230 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:11.231 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T01:24:11.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.236 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.237 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T01:24:11.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.244 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-06-03T01:24:11.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.246 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.249 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.255 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.261 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-06-03T01:24:11.263 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-06-03T01:24:11.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.267 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-06-03T01:24:11.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.273 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-06-03T01:24:11.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-06-03T01:24:11.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.277 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.281 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.281 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.284 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T01:24:11.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.286 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-06-03T01:24:11.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.287 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.290 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-06-03T01:24:11.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-06-03T01:24:11.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T01:24:11.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.293 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.297 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.303 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-06-03T01:24:11.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.312 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-06-03T01:24:11.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.315 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.322 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.324 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.329 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-06-03T01:24:11.330 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:11.332 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-06-03T01:24:11.333 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.334 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.336 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.338 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.340 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.342 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.343 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.346 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.347 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.348 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.349 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.351 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.352 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.356 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-06-03T01:24:11.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.360 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.364 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.364 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.364 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.364 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.364 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.365 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.367 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.367 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.367 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.367 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.368 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-06-03T01:24:11.369 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.370 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.371 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.374 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.375 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.377 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.378 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-06-03T01:24:11.381 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.381 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.381 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-06-03T01:24:11.381 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.383 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.384 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.385 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.386 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.389 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.393 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.394 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.395 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.398 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.400 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.402 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.403 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.404 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.405 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.407 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.407 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.407 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.407 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.407 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.411 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.412 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.413 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.418 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.418 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.419 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.423 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.424 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.425 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-06-03T01:24:11.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-06-03T01:24:11.427 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-06-03T01:24:11.428 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.428 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-06-03T01:24:11.428 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.428 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.430 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.434 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.435 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-06-03T01:24:11.438 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.439 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.440 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.441 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.442 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.444 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.449 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.450 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.451 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.453 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.453 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.453 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.453 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.454 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.456 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:11.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.458 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.461 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.465 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.468 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.469 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.473 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.478 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.480 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.483 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.486 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.487 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.491 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-06-03T01:24:11.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.493 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.494 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:11.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:11.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-06-03T01:24:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-06-03T01:24:11.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.498 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.499 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.502 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.503 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.504 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.507 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.508 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.509 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.510 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.512 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-06-03T01:24:11.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-06-03T01:24:11.517 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.519 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.522 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.523 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.527 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-06-03T01:24:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-06-03T01:24:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-06-03T01:24:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-06-03T01:24:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-06-03T01:24:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-06-03T01:24:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-06-03T01:24:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-06-03T01:24:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-06-03T01:24:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-06-03T01:24:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-06-03T01:24:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-06-03T01:24:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-06-03T01:24:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 25210, 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "z", 2024-06-03T01:24:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-06-03T01:24:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-06-03T01:24:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-06-03T01:24:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T01:24:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-06-03T01:24:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T01:24:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-06-03T01:24:11.743 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.745 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.747 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.752 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.754 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-06-03T01:24:11.755 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.756 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-06-03T01:24:11.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T01:24:11.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.761 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.762 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-06-03T01:24:11.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T01:24:11.764 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.765 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.767 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-06-03T01:24:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.769 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.770 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.772 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.774 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.776 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.776 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-06-03T01:24:11.776 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-06-03T01:24:11.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.778 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-06-03T01:24:11.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T01:24:11.781 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.782 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.782 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.782 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.782 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.783 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.786 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-06-03T01:24:11.787 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.789 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.790 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.791 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.794 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.795 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-06-03T01:24:11.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.797 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.798 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.799 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.800 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-06-03T01:24:11.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:11.803 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.804 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.806 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-06-03T01:24:11.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.811 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T01:24:11.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T01:24:11.813 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.815 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.816 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.818 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.819 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-06-03T01:24:11.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T01:24:11.821 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.823 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.824 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.825 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T01:24:11.827 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.827 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.827 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.827 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.828 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.831 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.832 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.834 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.836 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T01:24:11.837 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.838 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-06-03T01:24:11.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.843 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.844 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-06-03T01:24:11.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-06-03T01:24:11.846 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-06-03T01:24:11.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T01:24:11.849 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-06-03T01:24:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.853 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.854 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.855 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.857 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-06-03T01:24:11.858 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-06-03T01:24:11.859 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.860 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.862 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.864 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.865 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.866 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-06-03T01:24:11.867 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-06-03T01:24:11.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.867 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.867 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.867 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.867 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-06-03T01:24:11.868 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-06-03T01:24:11.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.869 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.869 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.869 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.869 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.869 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.871 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.872 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.872 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.872 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.872 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.872 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.872 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.874 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.876 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.877 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.879 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.882 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T01:24:11.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.885 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.888 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-06-03T01:24:11.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T01:24:11.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.890 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.890 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.890 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.890 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.891 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.895 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T01:24:11.896 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T01:24:11.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.897 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.897 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.897 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.897 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T01:24:11.898 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.900 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.901 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T01:24:11.902 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T01:24:11.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.902 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.902 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.902 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.902 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T01:24:11.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.904 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.905 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T01:24:11.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.909 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.910 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.911 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.912 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.913 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.914 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T01:24:11.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.916 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.919 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T01:24:11.919 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T01:24:11.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.919 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.919 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.920 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.921 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T01:24:11.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.923 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.923 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.923 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.923 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.923 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.923 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.923 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.925 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.926 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.927 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.928 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T01:24:11.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.930 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T01:24:11.934 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.935 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.936 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T01:24:11.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T01:24:11.944 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.945 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.948 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T01:24:11.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.950 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.952 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:11.953 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.954 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.955 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.956 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.957 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.958 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.961 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-06-03T01:24:11.962 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.963 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.966 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.968 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.970 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.971 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.974 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.976 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:11.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.978 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.978 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-06-03T01:24:11.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.978 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.978 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.978 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.980 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.981 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.982 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.983 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.984 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-06-03T01:24:11.985 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-06-03T01:24:11.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.988 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:11.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-06-03T01:24:11.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-06-03T01:24:11.991 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.993 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.995 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:11.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:11.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.995 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.995 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:11.995 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:11.996 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:11.997 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:11.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:11.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.000 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T01:24:12.001 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.003 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.004 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.005 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.007 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.008 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.009 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-06-03T01:24:12.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T01:24:12.011 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-06-03T01:24:12.012 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.013 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.014 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.015 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.016 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.017 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.018 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.019 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.019 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.019 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.019 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.019 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.019 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-06-03T01:24:12.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.022 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.027 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.029 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.031 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.033 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.034 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.035 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.038 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.039 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.040 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.041 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.042 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.044 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.045 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-06-03T01:24:12.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-06-03T01:24:12.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.050 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-06-03T01:24:12.051 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.052 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.053 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.054 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.054 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.054 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.054 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.054 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.056 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.057 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.058 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.060 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.061 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.062 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.063 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.064 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.065 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.067 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-06-03T01:24:12.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.070 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.071 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.074 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.075 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.077 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.078 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.079 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.080 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.081 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.084 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.085 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.086 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.087 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.088 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.089 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.090 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.092 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.093 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-06-03T01:24:12.094 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-06-03T01:24:12.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.095 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.095 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-06-03T01:24:12.095 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.095 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-06-03T01:24:12.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-06-03T01:24:12.097 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.099 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.100 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.101 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.105 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.108 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.110 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.114 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.115 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.116 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.118 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.120 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.124 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.127 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.129 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.130 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.131 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.132 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.134 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.135 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.136 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-06-03T01:24:12.138 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.141 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.143 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.145 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.147 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.150 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.152 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.154 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.158 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.159 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.160 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.163 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.165 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-06-03T01:24:12.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.169 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.170 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-06-03T01:24:12.170 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.170 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.170 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.170 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.174 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.175 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.176 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.177 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.178 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.179 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.183 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.185 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.186 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-06-03T01:24:12.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T01:24:12.188 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.189 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.190 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.193 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.194 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T01:24:12.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.199 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.201 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-06-03T01:24:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-06-03T01:24:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-06-03T01:24:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-06-03T01:24:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-06-03T01:24:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-06-03T01:24:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-06-03T01:24:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-06-03T01:24:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-06-03T01:24:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T01:24:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-06-03T01:24:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-06-03T01:24:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-06-03T01:24:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-06-03T01:24:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.372 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-06-03T01:24:12.374 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.374 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.374 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.374 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.375 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.377 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.378 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.382 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.385 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.386 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T01:24:12.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.389 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.390 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.392 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.394 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.397 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.399 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.401 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T01:24:12.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.407 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard", 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: "restful", 2024-06-03T01:24:12.408 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_schedule" 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:12.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:12.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.410 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:12.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.412 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.413 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.414 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.415 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.415 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.417 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.419 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.420 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.421 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.422 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-06-03T01:24:12.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-06-03T01:24:12.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.429 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.431 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.435 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.436 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.437 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.438 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-06-03T01:24:12.438 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T01:24:12.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.438 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.438 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-06-03T01:24:12.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-06-03T01:24:12.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:12.439 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.439 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.440 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-06-03T01:24:12.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-06-03T01:24:12.445 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.446 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:12.447 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.450 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.451 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.452 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.454 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.455 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.456 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-06-03T01:24:12.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-03T01:24:12.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-06-03T01:24:12.460 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-06-03T01:24:12.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.463 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.464 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:12.465 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-06-03T01:24:12.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-03T01:24:12.467 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-06-03T01:24:12.468 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.470 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.471 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-06-03T01:24:12.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-03T01:24:12.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-06-03T01:24:12.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-06-03T01:24:12.478 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-06-03T01:24:12.479 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-06-03T01:24:12.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-03T01:24:12.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.483 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-06-03T01:24:12.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-03T01:24:12.490 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.491 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-03T01:24:12.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-06-03T01:24:12.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-06-03T01:24:12.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.502 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-06-03T01:24:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-06-03T01:24:12.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-03T01:24:12.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.510 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-06-03T01:24:12.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.515 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-06-03T01:24:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.519 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.520 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.521 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-06-03T01:24:12.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.523 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.527 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.530 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-06-03T01:24:12.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-03T01:24:12.532 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.533 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.534 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.535 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.538 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.539 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-06-03T01:24:12.542 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-06-03T01:24:12.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.542 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.544 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.549 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.550 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.552 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.553 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-03T01:24:12.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.554 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-06-03T01:24:12.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.560 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-03T01:24:12.560 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.560 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.561 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-06-03T01:24:12.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.562 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.567 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-06-03T01:24:12.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.569 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-06-03T01:24:12.569 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-03T01:24:12.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.573 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.574 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.579 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.581 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-06-03T01:24:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-06-03T01:24:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.584 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-03T01:24:12.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.587 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.590 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-06-03T01:24:12.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-06-03T01:24:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-03T01:24:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.596 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-03T01:24:12.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-03T01:24:12.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.606 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.607 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-03T01:24:12.608 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.610 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.611 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.612 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.613 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.614 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-03T01:24:12.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.616 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.618 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.619 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.621 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-03T01:24:12.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-03T01:24:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.623 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.625 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.626 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.629 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.630 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-03T01:24:12.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-03T01:24:12.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.635 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.638 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.642 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.643 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.646 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-03T01:24:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.649 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.651 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-06-03T01:24:12.652 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.653 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:12.656 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.657 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.658 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-03T01:24:12.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.661 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-06-03T01:24:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0b8c0a2d00 msgr2=0x7f0b8c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:12.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0b8c0a2d00 0x7f0b8c0a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fc0 tx=0x7f0b98005f90 comp rx=0 tx=0).stop 2024-06-03T01:24:12.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 msgr2=0x7f0ba81a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:12.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba81a4b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0b90006010 tx=0x7f0b90004040 comp rx=0 tx=0).stop 2024-06-03T01:24:12.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 shutdown_connections 2024-06-03T01:24:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f0b8c0a2d00 0x7f0b8c0a51c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ba8105d40 0x7f0ba819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ba8101340 0x7f0ba81a4b40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 --2- 172.21.15.94:0/704877772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ba8100a20 0x7f0ba81a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 >> 172.21.15.94:0/704877772 conn(0x7f0ba80779a0 msgr2=0x7f0ba81037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 shutdown_connections 2024-06-03T01:24:12.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:10.981+0000 7f0bae800640 1 -- 172.21.15.94:0/704877772 wait complete. 2024-06-03T01:24:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-06-03T01:24:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-06-03T01:24:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-06-03T01:24:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-06-03T01:24:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-06-03T01:24:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-06-03T01:24:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-03T01:24:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-06-03T01:24:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-03T01:24:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-06-03T01:24:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-06-03T01:24:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-06-03T01:24:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-06-03T01:24:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-06-03T01:24:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-03T01:24:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-06-03T01:24:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-03T01:24:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-06-03T01:24:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-06-03T01:24:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-06-03T01:24:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T01:24:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-06-03T01:24:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-03T01:24:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-06-03T01:24:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-06-03T01:24:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.766 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.769 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.771 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.773 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.774 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.775 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.777 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-06-03T01:24:12.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.780 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.782 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.782 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-06-03T01:24:12.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.782 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.782 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-03T01:24:12.782 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.783 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.784 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.786 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.788 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.789 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.790 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.792 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.794 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.795 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.796 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.796 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.796 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.796 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.797 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.798 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.798 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.798 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.799 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.800 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.801 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.802 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.803 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-06-03T01:24:12.804 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-06-03T01:24:12.805 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.806 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.808 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.809 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.810 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-03T01:24:12.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-03T01:24:12.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.812 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.814 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.816 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.817 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.819 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.819 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.819 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.819 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.819 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.822 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.823 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.824 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.825 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.826 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.827 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.828 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.829 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.831 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.832 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.835 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.836 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.837 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.838 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.842 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.844 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.846 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.848 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-06-03T01:24:12.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-06-03T01:24:12.850 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.852 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.853 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.854 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.855 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.856 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.857 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.859 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-06-03T01:24:12.860 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.861 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.862 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.863 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.864 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.866 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.869 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.870 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.871 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.872 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.873 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.874 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.875 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.876 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.877 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.878 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-06-03T01:24:12.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-03T01:24:12.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.881 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.882 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-06-03T01:24:12.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.885 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.886 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.887 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.889 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.890 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.891 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.892 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.893 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.894 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.895 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-06-03T01:24:12.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.898 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.899 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.900 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.901 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.902 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.904 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.905 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.906 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-03T01:24:12.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.911 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.912 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.913 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-06-03T01:24:12.914 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.915 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.916 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.917 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.919 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.921 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.922 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.923 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.924 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.925 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.926 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.926 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.926 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.926 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.926 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.927 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.928 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.929 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.930 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.931 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.931 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.931 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-06-03T01:24:12.932 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.933 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.934 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.936 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-06-03T01:24:12.938 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.940 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.942 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.943 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.944 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.945 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.947 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.949 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-06-03T01:24:12.950 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.951 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.952 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.954 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.955 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.956 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.957 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.958 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.960 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.962 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.963 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.965 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.966 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.968 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.969 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.971 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-06-03T01:24:12.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:12.974 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.975 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.976 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:12.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.978 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.979 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:12.980 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:12.981 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.983 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.984 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-06-03T01:24:12.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-06-03T01:24:12.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.987 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-06-03T01:24:12.991 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.992 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.994 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:12.995 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.996 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.997 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:12.998 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:12.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.000 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-06-03T01:24:13.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.002 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.003 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.008 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.009 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.010 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.011 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.012 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.012 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.012 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.012 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.013 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.015 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-06-03T01:24:13.016 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.017 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.018 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.019 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.020 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.021 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.021 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.022 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.024 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.026 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.027 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.028 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.028 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.028 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.028 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.028 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.028 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.028 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.029 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.030 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.030 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.030 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.030 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.030 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.030 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.034 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.034 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.034 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.035 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.037 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-06-03T01:24:13.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.040 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.041 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.043 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.044 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.048 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.052 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.054 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.056 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-06-03T01:24:13.057 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.058 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.059 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.061 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.062 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.063 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.066 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-06-03T01:24:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-06-03T01:24:13.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:13.070 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-06-03T01:24:13.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.070 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.072 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.073 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.074 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.075 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.076 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.079 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.080 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.085 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.086 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.086 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.086 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.086 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.087 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-06-03T01:24:13.088 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.089 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.090 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.091 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.091 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.091 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.091 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.091 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.094 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.096 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.099 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.101 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.102 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.103 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-06-03T01:24:13.106 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.107 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.109 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-06-03T01:24:13.110 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.111 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.114 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-06-03T01:24:13.114 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-06-03T01:24:13.114 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-06-03T01:24:13.114 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-06-03T01:24:13.114 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-06-03T01:24:13.114 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-06-03T01:24:13.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.116 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.118 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.119 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.119 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.119 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.119 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.119 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.120 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.125 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-06-03T01:24:13.125 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-06-03T01:24:13.125 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-06-03T01:24:13.125 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-06-03T01:24:13.125 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-06-03T01:24:13.125 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-06-03T01:24:13.125 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-06-03T01:24:13.126 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.127 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.129 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-06-03T01:24:13.130 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.131 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-06-03T01:24:13.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.133 INFO:teuthology.orchestra.run.smithi094.stdout: "services": { 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard": "https://172.21.15.94:8443/" 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: "always_on_modules": { 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: "octopus": [ 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-06-03T01:24:13.134 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "pacific": [ 2024-06-03T01:24:13.135 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-06-03T01:24:13.136 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "quincy": [ 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-06-03T01:24:13.137 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: "reef": [ 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-06-03T01:24:13.138 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-06-03T01:24:13.139 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "squid": [ 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-06-03T01:24:13.140 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: "last_failure_osd_epoch": 101, 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: "active_clients": [ 2024-06-03T01:24:13.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 84506305 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.142 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "libcephsqlite", 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:24:13.143 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3858866561 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-06-03T01:24:13.144 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1008027704 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-06-03T01:24:13.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 711458770 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.146 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1430716103 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:13.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2121925738 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.148 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:13.149 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-06-03T01:24:13.149 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:24:13.156 DEBUG:tasks.check_counter:Getting stats from a 2024-06-03T01:24:13.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-03T01:24:13.157 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:11 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/44755956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T01:24:13.157 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:11 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/704877772' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T01:24:13.157 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:12 smithi094 ceph-mon[32119]: pgmap v434: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 6.5 MiB/s wr, 806 op/s 2024-06-03T01:24:13.158 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:11 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/44755956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T01:24:13.158 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:11 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/704877772' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T01:24:13.158 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:12 smithi107 ceph-mon[36622]: pgmap v434: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 6.5 MiB/s wr, 806 op/s 2024-06-03T01:24:13.158 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:11 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/44755956' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-03T01:24:13.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:11 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/704877772' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-03T01:24:13.159 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:12 smithi179 ceph-mon[37041]: pgmap v434: 129 pgs: 129 active+clean; 164 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 6.5 MiB/s wr, 806 op/s 2024-06-03T01:24:13.674 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:24:14.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:14 smithi094 ceph-mon[32119]: pgmap v435: 129 pgs: 129 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 28 MiB/s wr, 870 op/s 2024-06-03T01:24:14.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:14 smithi107 ceph-mon[36622]: pgmap v435: 129 pgs: 129 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 28 MiB/s wr, 870 op/s 2024-06-03T01:24:14.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:14 smithi179 ceph-mon[37041]: pgmap v435: 129 pgs: 129 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 28 MiB/s wr, 870 op/s 2024-06-03T01:24:15.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:15 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:15.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:15 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:15.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:15 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:16.553 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:24:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1719, 2024-06-03T01:24:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1663, 2024-06-03T01:24:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 6559544, 2024-06-03T01:24:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 619822, 2024-06-03T01:24:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-06-03T01:24:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-06-03T01:24:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.468191735, 2024-06-03T01:24:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.172837859, 2024-06-03T01:24:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.185178091, 2024-06-03T01:24:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.098543629, 2024-06-03T01:24:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1663, 2024-06-03T01:24:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.182316924, 2024-06-03T01:24:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000109631 2024-06-03T01:24:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1937152, 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 199744 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1070, 2024-06-03T01:24:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1308, 2024-06-03T01:24:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 10901010, 2024-06-03T01:24:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 901870, 2024-06-03T01:24:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-06-03T01:24:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 5, 2024-06-03T01:24:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.330505937, 2024-06-03T01:24:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.114452527, 2024-06-03T01:24:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.114198004, 2024-06-03T01:24:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.071949642, 2024-06-03T01:24:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1305, 2024-06-03T01:24:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.133145899, 2024-06-03T01:24:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000102027 2024-06-03T01:24:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 562496 2024-06-03T01:24:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1111, 2024-06-03T01:24:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1113, 2024-06-03T01:24:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 12868083, 2024-06-03T01:24:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 353299, 2024-06-03T01:24:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-06-03T01:24:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-06-03T01:24:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.297087284, 2024-06-03T01:24:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.092644983, 2024-06-03T01:24:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.114421999, 2024-06-03T01:24:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.065227804, 2024-06-03T01:24:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1111, 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.101692010, 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000091531 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-03T01:24:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-06-03T01:24:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-03T01:24:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-06-03T01:24:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-03T01:24:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-03T01:24:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5772, 2024-06-03T01:24:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.324917950, 2024-06-03T01:24:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000056292 2024-06-03T01:24:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-03T01:24:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-03T01:24:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-06-03T01:24:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-06-03T01:24:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-06-03T01:24:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-06-03T01:24:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-06-03T01:24:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-06-03T01:24:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-06-03T01:24:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 2477, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 11, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 2466, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-06-03T01:24:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 2462, 2024-06-03T01:24:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-06-03T01:24:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-06-03T01:24:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-06-03T01:24:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 5, 2024-06-03T01:24:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-06-03T01:24:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-06-03T01:24:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-06-03T01:24:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-06-03T01:24:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-06-03T01:24:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-06-03T01:24:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-06-03T01:24:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-06-03T01:24:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-06-03T01:24:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-06-03T01:24:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-06-03T01:24:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-06-03T01:24:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-06-03T01:24:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-06-03T01:24:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-06-03T01:24:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T01:24:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-06-03T01:24:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-06-03T01:24:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-06-03T01:24:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-06-03T01:24:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-06-03T01:24:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-06-03T01:24:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-06-03T01:24:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-06-03T01:24:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-06-03T01:24:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-06-03T01:24:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-06-03T01:24:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-06-03T01:24:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-06-03T01:24:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-06-03T01:24:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-06-03T01:24:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-06-03T01:24:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-06-03T01:24:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-06-03T01:24:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-06-03T01:24:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-06-03T01:24:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-06-03T01:24:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 17861, 2024-06-03T01:24:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-06-03T01:24:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-06-03T01:24:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-06-03T01:24:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-06-03T01:24:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-06-03T01:24:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 21, 2024-06-03T01:24:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 3, 2024-06-03T01:24:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-06-03T01:24:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-06-03T01:24:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-06-03T01:24:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 31603014, 2024-06-03T01:24:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 31595250, 2024-06-03T01:24:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-06-03T01:24:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 17861 2024-06-03T01:24:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-06-03T01:24:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 103, 2024-06-03T01:24:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 2512, 2024-06-03T01:24:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 2409, 2024-06-03T01:24:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 24, 2024-06-03T01:24:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 26, 2024-06-03T01:24:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 2, 2024-06-03T01:24:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 2477, 2024-06-03T01:24:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 2540, 2024-06-03T01:24:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 63, 2024-06-03T01:24:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-06-03T01:24:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 54392, 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215420 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-06-03T01:24:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-06-03T01:24:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-06-03T01:24:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-06-03T01:24:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-06-03T01:24:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-06-03T01:24:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-06-03T01:24:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.589 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.589 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.589 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-06-03T01:24:16.589 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.589 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.589 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-06-03T01:24:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-06-03T01:24:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-06-03T01:24:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-06-03T01:24:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-06-03T01:24:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-06-03T01:24:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-06-03T01:24:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-06-03T01:24:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-06-03T01:24:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-06-03T01:24:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-06-03T01:24:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-06-03T01:24:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-06-03T01:24:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-06-03T01:24:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-06-03T01:24:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 1, 2024-06-03T01:24:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 3, 2024-06-03T01:24:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 2, 2024-06-03T01:24:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-06-03T01:24:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-06-03T01:24:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-06-03T01:24:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-06-03T01:24:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 2324, 2024-06-03T01:24:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-03T01:24:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-03T01:24:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 211929, 2024-06-03T01:24:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 154, 2024-06-03T01:24:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-03T01:24:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-03T01:24:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 181, 2024-06-03T01:24:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-03T01:24:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 2063277, 2024-06-03T01:24:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 12897, 2024-06-03T01:24:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-03T01:24:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-03T01:24:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-06-03T01:24:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-06-03T01:24:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-06-03T01:24:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 3499, 2024-06-03T01:24:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-06-03T01:24:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-06-03T01:24:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 3499, 2024-06-03T01:24:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-06-03T01:24:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3499, 2024-06-03T01:24:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3.355945214, 2024-06-03T01:24:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000959115 2024-06-03T01:24:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-06-03T01:24:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-06-03T01:24:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3499, 2024-06-03T01:24:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3500 2024-06-03T01:24:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 3499, 2024-06-03T01:24:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 3499, 2024-06-03T01:24:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-06-03T01:24:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-06-03T01:24:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-06-03T01:24:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 2042, 2024-06-03T01:24:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-06-03T01:24:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 1455, 2024-06-03T01:24:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-06-03T01:24:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-06-03T01:24:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-06-03T01:24:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-06-03T01:24:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-06-03T01:24:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-06-03T01:24:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-06-03T01:24:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-06-03T01:24:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-06-03T01:24:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-06-03T01:24:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-06-03T01:24:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-06-03T01:24:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-06-03T01:24:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-06-03T01:24:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-06-03T01:24:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-06-03T01:24:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-06-03T01:24:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 117, 2024-06-03T01:24:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-06-03T01:24:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 19, 2024-06-03T01:24:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 7, 2024-06-03T01:24:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 8, 2024-06-03T01:24:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 1, 2024-06-03T01:24:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-06-03T01:24:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-06-03T01:24:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-06-03T01:24:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-06-03T01:24:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-06-03T01:24:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-06-03T01:24:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-06-03T01:24:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-06-03T01:24:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-06-03T01:24:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-06-03T01:24:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-06-03T01:24:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T01:24:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-06-03T01:24:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-06-03T01:24:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 3896, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 29954960, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 3896, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3896, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 29954960, 2024-06-03T01:24:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-03T01:24:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3499, 2024-06-03T01:24:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 27413689, 2024-06-03T01:24:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3499, 2024-06-03T01:24:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 27413689, 2024-06-03T01:24:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-06-03T01:24:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3499, 2024-06-03T01:24:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 3499, 2024-06-03T01:24:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3499, 2024-06-03T01:24:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 3499, 2024-06-03T01:24:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-03T01:24:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-03T01:24:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x5651e36da780": { 2024-06-03T01:24:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-03T01:24:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-03T01:24:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-03T01:24:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:16.644 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:24:16.645 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:16 smithi094 ceph-mon[32119]: pgmap v436: 129 pgs: 129 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 25 MiB/s wr, 766 op/s 2024-06-03T01:24:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:16 smithi107 ceph-mon[36622]: pgmap v436: 129 pgs: 129 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 25 MiB/s wr, 766 op/s 2024-06-03T01:24:16.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:16 smithi179 ceph-mon[37041]: pgmap v436: 129 pgs: 129 active+clean; 357 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 25 MiB/s wr, 766 op/s 2024-06-03T01:24:17.953 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-03T01:24:17.953 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-03T01:24:17.953 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-06-03T01:24:17.953 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:17.954 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:17.954 DEBUG:tasks.check_counter:Getting stats from d 2024-06-03T01:24:17.954 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-03T01:24:18.412 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:24:18.773 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:18 smithi094 ceph-mon[32119]: pgmap v437: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 20 MiB/s wr, 226 op/s 2024-06-03T01:24:18.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:18 smithi107 ceph-mon[36622]: pgmap v437: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 20 MiB/s wr, 226 op/s 2024-06-03T01:24:18.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:18 smithi179 ceph-mon[37041]: pgmap v437: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 20 MiB/s wr, 226 op/s 2024-06-03T01:24:20.449 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:20 smithi094 ceph-mon[32119]: pgmap v438: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 19 MiB/s wr, 219 op/s 2024-06-03T01:24:20.764 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:24:20.764 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:20.764 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-06-03T01:24:20.764 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-06-03T01:24:20.764 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.003931121, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000897500, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T01:24:20.765 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:20.766 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:20.766 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:20.766 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:20.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.766 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:20.766 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:20.766 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-06-03T01:24:20.767 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 246, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 198992, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.050301493, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.038087126, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.004371039, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T01:24:20.768 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:20.769 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 243, 2024-06-03T01:24:20.769 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.038256273, 2024-06-03T01:24:20.769 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000157433 2024-06-03T01:24:20.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.769 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:20.769 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 198992 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:20.770 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 338, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 325, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1615680, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 200048, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.140865006, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.051903474, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.094699251, 2024-06-03T01:24:20.771 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.015728162, 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 325, 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.032675394, 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000100539 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:20.772 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1615680, 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 200048 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.773 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:20.774 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:20.775 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:20.776 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:20.777 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:20.777 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:20.777 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:20.777 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:20.777 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:20.777 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:20.778 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-03T01:24:20.778 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:20.778 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:20.778 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:20.778 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:20.778 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 149745, 2024-06-03T01:24:20.778 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 31, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:20.779 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-06-03T01:24:20.780 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:20.781 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-03T01:24:20.781 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:20.781 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-03T01:24:20.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:20.781 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:20.781 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:20.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-03T01:24:20.782 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:20.782 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 336, 2024-06-03T01:24:20.782 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1565521, 2024-06-03T01:24:20.783 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:20.783 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 336, 2024-06-03T01:24:20.783 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:20.783 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:20.783 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 336, 2024-06-03T01:24:20.783 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1565521, 2024-06-03T01:24:20.783 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:20.784 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:20.784 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:20.784 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:20.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:20.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:20.785 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:24:20.785 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:20 smithi107 ceph-mon[36622]: pgmap v438: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 19 MiB/s wr, 219 op/s 2024-06-03T01:24:20.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:20 smithi179 ceph-mon[37041]: pgmap v438: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 19 MiB/s wr, 219 op/s 2024-06-03T01:24:21.712 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-03T01:24:21.712 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-03T01:24:21.712 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-06-03T01:24:21.713 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-06-03T01:24:21.713 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-06-03T01:24:21.713 DEBUG:tasks.check_counter:Getting stats from g 2024-06-03T01:24:21.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-03T01:24:22.168 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:24:22.591 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:22 smithi094 ceph-mon[32119]: pgmap v439: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 17 MiB/s wr, 189 op/s 2024-06-03T01:24:22.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:22 smithi107 ceph-mon[36622]: pgmap v439: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 17 MiB/s wr, 189 op/s 2024-06-03T01:24:22.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:22 smithi179 ceph-mon[37041]: pgmap v439: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 17 MiB/s wr, 189 op/s 2024-06-03T01:24:24.579 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:24 smithi094 ceph-mon[32119]: pgmap v440: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 17 MiB/s wr, 194 op/s 2024-06-03T01:24:24.681 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:24:24.682 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:24.682 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1644, 2024-06-03T01:24:24.682 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1586, 2024-06-03T01:24:24.682 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 2939170, 2024-06-03T01:24:24.682 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 596286, 2024-06-03T01:24:24.682 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-06-03T01:24:24.682 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.463728540, 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.196063106, 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.169828953, 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.090242344, 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1586, 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.160700468, 2024-06-03T01:24:24.683 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000101324 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1944752, 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 201648 2024-06-03T01:24:24.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2222, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2464, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 13147470, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 1272273, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 5, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.496856084, 2024-06-03T01:24:24.685 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.171216747, 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.170047069, 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.116355173, 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2461, 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.237508881, 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000096509 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.686 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 566256 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:24.687 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 6, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1629, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 1284, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.006305443, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000164464, 2024-06-03T01:24:24.688 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.005415445, 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000123482, 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4, 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.005197600, 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001299400 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.689 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-06-03T01:24:24.690 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5848, 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.303975158, 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000051979 2024-06-03T01:24:24.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-06-03T01:24:24.694 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 747, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 747, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 747, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-06-03T01:24:24.695 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 1, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-06-03T01:24:24.696 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-06-03T01:24:24.697 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:24.698 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:24.699 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-06-03T01:24:24.700 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-06-03T01:24:24.701 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-06-03T01:24:24.701 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-06-03T01:24:24.701 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:24.701 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:24.701 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-06-03T01:24:24.701 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-06-03T01:24:24.701 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:24.702 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.703 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 10374, 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-06-03T01:24:24.704 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-06-03T01:24:24.705 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-06-03T01:24:24.705 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-06-03T01:24:24.705 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 13, 2024-06-03T01:24:24.705 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-06-03T01:24:24.705 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-06-03T01:24:24.705 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-06-03T01:24:24.705 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 17367104, 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 17359340, 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.706 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 10374 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 13, 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 922, 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 909, 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 11, 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 28, 2024-06-03T01:24:24.707 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 17, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 747, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 943, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 196, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 52112, 2024-06-03T01:24:24.708 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215420 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-06-03T01:24:24.709 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-06-03T01:24:24.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.710 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-06-03T01:24:24.710 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.710 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-06-03T01:24:24.711 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.713 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.714 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.716 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-06-03T01:24:24.716 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.716 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.716 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.716 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-06-03T01:24:24.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.717 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.718 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-06-03T01:24:24.719 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.720 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.722 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.723 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.725 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.726 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.728 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 4, 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 2, 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 3, 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-06-03T01:24:24.729 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 3074, 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:24.730 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:24.731 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:24.732 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:24.733 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:24.734 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 200056, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 59, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:24.735 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 181, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-03T01:24:24.736 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 604552, 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 3173, 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-06-03T01:24:24.737 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 3465, 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 3465, 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3465, 2024-06-03T01:24:24.738 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3.537942256, 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001021051 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3465, 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 3466 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.739 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 3465, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 3465, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 2060, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 1403, 2024-06-03T01:24:24.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:24.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-06-03T01:24:24.742 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-06-03T01:24:24.743 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-06-03T01:24:24.744 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 117, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 19, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 6, 2024-06-03T01:24:24.745 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 8, 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 2, 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-06-03T01:24:24.746 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-06-03T01:24:24.747 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.748 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 3866, 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 15717091, 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 3866, 2024-06-03T01:24:24.749 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3866, 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 15717091, 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:24.751 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3465, 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 13173787, 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3465, 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 13173787, 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.752 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:24.753 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 3465, 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 3465, 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 3465, 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 3465, 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:24.754 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-03T01:24:24.755 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-03T01:24:24.756 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x55e46df08780": { 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:24.757 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-06-03T01:24:24.758 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-06-03T01:24:24.759 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:24.759 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:24.759 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:24.759 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:24.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:24.759 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:24:24.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:24 smithi107 ceph-mon[36622]: pgmap v440: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 17 MiB/s wr, 194 op/s 2024-06-03T01:24:24.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:24 smithi179 ceph-mon[37041]: pgmap v440: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 17 MiB/s wr, 194 op/s 2024-06-03T01:24:25.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:25 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:25.734 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-06-03T01:24:25.734 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-06-03T01:24:25.734 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-06-03T01:24:25.734 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:25.734 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:25.734 DEBUG:tasks.check_counter:Getting stats from j 2024-06-03T01:24:25.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-03T01:24:25.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:25 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:25.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:25 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:26.197 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:24:26.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:26 smithi094 ceph-mon[32119]: pgmap v441: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 93 op/s 2024-06-03T01:24:26.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:26 smithi107 ceph-mon[36622]: pgmap v441: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 93 op/s 2024-06-03T01:24:26.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:26 smithi179 ceph-mon[37041]: pgmap v441: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 246 KiB/s wr, 93 op/s 2024-06-03T01:24:28.531 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:28 smithi094 ceph-mon[32119]: pgmap v442: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 335 KiB/s wr, 101 op/s 2024-06-03T01:24:28.649 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-06-03T01:24:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-06-03T01:24:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-06-03T01:24:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-06-03T01:24:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-06-03T01:24:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.003625229, 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000780017, 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:28.650 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.651 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 341, 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 328, 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1616496, 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 201968, 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-06-03T01:24:28.652 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.145484731, 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.053018276, 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.102521469, 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.015985850, 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 328, 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.035905289, 2024-06-03T01:24:28.653 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000109467 2024-06-03T01:24:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:28.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1616496, 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 201968 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 249, 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-06-03T01:24:28.655 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 199520, 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.049743793, 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.038247246, 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.003854541, 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:28.656 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 246, 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.042885395, 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000174330 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.657 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 199520 2024-06-03T01:24:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-06-03T01:24:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-06-03T01:24:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-06-03T01:24:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.658 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:28.659 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:28.660 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:28.661 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:28.662 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148714, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-06-03T01:24:28.663 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:28.664 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-06-03T01:24:28.665 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 339, 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1565911, 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 339, 2024-06-03T01:24:28.666 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 339, 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1565911, 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-06-03T01:24:28.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:28.668 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-06-03T01:24:28.668 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-06-03T01:24:28.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:28 smithi107 ceph-mon[36622]: pgmap v442: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 335 KiB/s wr, 101 op/s 2024-06-03T01:24:28.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:28 smithi179 ceph-mon[37041]: pgmap v442: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 335 KiB/s wr, 101 op/s 2024-06-03T01:24:29.564 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-03T01:24:29.564 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-03T01:24:29.564 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-06-03T01:24:29.565 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-06-03T01:24:29.565 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-06-03T01:24:29.565 DEBUG:tasks.check_counter:Getting stats from b 2024-06-03T01:24:29.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-03T01:24:29.799 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:24:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:30 smithi107 ceph-mon[36622]: pgmap v443: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 13 op/s 2024-06-03T01:24:30.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:30 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:30.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:30 smithi094 ceph-mon[32119]: pgmap v443: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 13 op/s 2024-06-03T01:24:30.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:30 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:30 smithi179 ceph-mon[37041]: pgmap v443: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 13 op/s 2024-06-03T01:24:30.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:30 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:31.655 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-06-03T01:24:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 57774, 2024-06-03T01:24:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 51583, 2024-06-03T01:24:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 18647632, 2024-06-03T01:24:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 51979558, 2024-06-03T01:24:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 132, 2024-06-03T01:24:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 8, 2024-06-03T01:24:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 3.079754614, 2024-06-03T01:24:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 1.147964586, 2024-06-03T01:24:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 2.088544631, 2024-06-03T01:24:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.265636676, 2024-06-03T01:24:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 51585, 2024-06-03T01:24:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 4.474136170, 2024-06-03T01:24:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000086733 2024-06-03T01:24:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 92659, 2024-06-03T01:24:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.013731041, 2024-06-03T01:24:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000148 2024-06-03T01:24:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 2529504, 2024-06-03T01:24:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 382672 2024-06-03T01:24:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 84326, 2024-06-03T01:24:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 107380, 2024-06-03T01:24:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 26919309, 2024-06-03T01:24:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 134743098, 2024-06-03T01:24:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 21, 2024-06-03T01:24:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 7, 2024-06-03T01:24:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 5.074793309, 2024-06-03T01:24:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 2.146542456, 2024-06-03T01:24:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 3.150472251, 2024-06-03T01:24:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.410417811, 2024-06-03T01:24:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 107374, 2024-06-03T01:24:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 5.794706538, 2024-06-03T01:24:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000053967 2024-06-03T01:24:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 136483, 2024-06-03T01:24:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.024373425, 2024-06-03T01:24:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000178 2024-06-03T01:24:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 280656 2024-06-03T01:24:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 70160, 2024-06-03T01:24:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 76289, 2024-06-03T01:24:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 17922236, 2024-06-03T01:24:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 100131441, 2024-06-03T01:24:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 13, 2024-06-03T01:24:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 7, 2024-06-03T01:24:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 4.202719224, 2024-06-03T01:24:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 1.835456753, 2024-06-03T01:24:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 2.487915795, 2024-06-03T01:24:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.347909447, 2024-06-03T01:24:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 76287, 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3.058866626, 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000040096 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 119346, 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.017041496, 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000142 2024-06-03T01:24:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-03T01:24:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 394944 2024-06-03T01:24:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 37185, 2024-06-03T01:24:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 24.775462991, 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000666275 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6584, 2024-06-03T01:24:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1.015447683, 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000154229 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 36192, 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 30285, 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-06-03T01:24:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 30285, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 495.550222739, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.016362893 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 5907, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 5, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 1014, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 2, 2024-06-03T01:24:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-06-03T01:24:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 1050, 2024-06-03T01:24:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 49, 2024-06-03T01:24:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 136, 2024-06-03T01:24:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 865, 2024-06-03T01:24:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 980, 2024-06-03T01:24:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 3272, 2024-06-03T01:24:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 252, 2024-06-03T01:24:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 252, 2024-06-03T01:24:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 9, 2024-06-03T01:24:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 54103, 2024-06-03T01:24:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 41379, 2024-06-03T01:24:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 5891, 2024-06-03T01:24:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 151, 2024-06-03T01:24:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 145, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 145678, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 23, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 25, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 2315, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 13, 2024-06-03T01:24:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 515, 2024-06-03T01:24:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 590, 2024-06-03T01:24:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 204782854, 2024-06-03T01:24:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-06-03T01:24:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 13, 2024-06-03T01:24:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 12870, 2024-06-03T01:24:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 14107, 2024-06-03T01:24:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 19295, 2024-06-03T01:24:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 4610, 2024-06-03T01:24:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 1690, 2024-06-03T01:24:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 11885 2024-06-03T01:24:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-06-03T01:24:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 4, 2024-06-03T01:24:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-06-03T01:24:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-06-03T01:24:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 151, 2024-06-03T01:24:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 56, 2024-06-03T01:24:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 32, 2024-06-03T01:24:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 3159, 2024-06-03T01:24:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 3127, 2024-06-03T01:24:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-06-03T01:24:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-06-03T01:24:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-06-03T01:24:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 25, 2024-06-03T01:24:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-06-03T01:24:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 2, 2024-06-03T01:24:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-06-03T01:24:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-06-03T01:24:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-06-03T01:24:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 31712, 2024-06-03T01:24:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 29346, 2024-06-03T01:24:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 29346, 2024-06-03T01:24:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 2366, 2024-06-03T01:24:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-06-03T01:24:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 5748, 2024-06-03T01:24:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 109, 2024-06-03T01:24:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 106, 2024-06-03T01:24:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 106, 2024-06-03T01:24:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 3, 2024-06-03T01:24:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 1, 2024-06-03T01:24:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-06-03T01:24:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 8, 2024-06-03T01:24:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 168761271, 2024-06-03T01:24:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 175285458, 2024-06-03T01:24:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-06-03T01:24:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-06-03T01:24:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 19029, 2024-06-03T01:24:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 85.947708766, 2024-06-03T01:24:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004516669 2024-06-03T01:24:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-06-03T01:24:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-06-03T01:24:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 213, 2024-06-03T01:24:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 5538, 2024-06-03T01:24:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 5325, 2024-06-03T01:24:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 56, 2024-06-03T01:24:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 176, 2024-06-03T01:24:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 120, 2024-06-03T01:24:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 718, 2024-06-03T01:24:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 9334, 2024-06-03T01:24:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 8616, 2024-06-03T01:24:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 116, 2024-06-03T01:24:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 3964, 2024-06-03T01:24:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 3848, 2024-06-03T01:24:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 86964, 2024-06-03T01:24:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 231804 2024-06-03T01:24:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-06-03T01:24:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 53520, 2024-06-03T01:24:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 21440, 2024-06-03T01:24:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 36192, 2024-06-03T01:24:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 480, 2024-06-03T01:24:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 42177, 2024-06-03T01:24:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-06-03T01:24:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3780, 2024-06-03T01:24:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 18.831529924, 2024-06-03T01:24:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004981886 2024-06-03T01:24:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-06-03T01:24:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 12882, 2024-06-03T01:24:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 92.947730266, 2024-06-03T01:24:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.007215318 2024-06-03T01:24:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3164, 2024-06-03T01:24:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 314.741273777, 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.099475750 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000126819, 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000126819 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-06-03T01:24:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 14, 2024-06-03T01:24:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007906245, 2024-06-03T01:24:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000564731 2024-06-03T01:24:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-06-03T01:24:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 119, 2024-06-03T01:24:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.517970713, 2024-06-03T01:24:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004352695 2024-06-03T01:24:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-06-03T01:24:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-06-03T01:24:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 14, 2024-06-03T01:24:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1.683692427, 2024-06-03T01:24:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.120263744 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 10, 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.015222479, 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001522247 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-06-03T01:24:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6504, 2024-06-03T01:24:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 44.641784838, 2024-06-03T01:24:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.006863743 2024-06-03T01:24:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-06-03T01:24:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 547, 2024-06-03T01:24:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3.186322031, 2024-06-03T01:24:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005825085 2024-06-03T01:24:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-06-03T01:24:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1, 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000856955, 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000856955 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 8, 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.257752111, 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.032219013 2024-06-03T01:24:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-06-03T01:24:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001370600, 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000685300 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 72, 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.306570356, 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004257921 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 8, 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.022550070, 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002818758 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3159, 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 18.387563128, 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005820691 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 5, 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 3, 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 2, 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 8459, 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 4229, 2024-06-03T01:24:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 829, 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 360, 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 360, 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 313495, 2024-06-03T01:24:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 670, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 88, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 1, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 181, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 3456988, 2024-06-03T01:24:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 37435, 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-06-03T01:24:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 33425, 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 179329466, 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 37, 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 33425, 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 33425, 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 810.122844783, 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.024237033 2024-06-03T01:24:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-06-03T01:24:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-06-03T01:24:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 33425, 2024-06-03T01:24:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 43966 2024-06-03T01:24:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 33425, 2024-06-03T01:24:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 202, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 33223, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 867, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 5313, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 193, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 19222, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 382, 2024-06-03T01:24:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 2, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 6845, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 2, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 8080, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-06-03T01:24:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-06-03T01:24:31.741 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-06-03T01:24:31.742 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 117, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 28, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 12, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 12, 2024-06-03T01:24:31.743 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 2650, 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 14, 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 396 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-06-03T01:24:31.744 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:31.745 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 10, 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 5, 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 6254, 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 3127, 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:31.746 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 212256, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 44303372, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 212256, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:31.747 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 212256, 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 44303372, 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:31.749 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 33425, 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 179672561, 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 33425, 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 179672561, 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:31.751 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 33425, 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 33425, 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 33425, 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 33425, 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.752 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3127, 2024-06-03T01:24:31.753 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 343299, 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3127, 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 193, 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 343299, 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:31.754 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55e5e4036500": { 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:31.755 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 31712, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 171091154, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 31712, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 19029, 2024-06-03T01:24:31.756 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 171091154, 2024-06-03T01:24:31.757 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:31.757 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:31.757 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:31.757 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:31.757 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.757 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:31.757 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-06-03T01:24:32.527 INFO:tasks.check_counter:Daemon mds.b mds.exported=25 2024-06-03T01:24:32.527 INFO:tasks.check_counter:Daemon mds.b mds.imported=13 2024-06-03T01:24:32.527 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-06-03T01:24:32.527 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=14 2024-06-03T01:24:32.528 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=8 2024-06-03T01:24:32.528 DEBUG:tasks.check_counter:Getting stats from e 2024-06-03T01:24:32.528 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-03T01:24:32.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:32 smithi107 ceph-mon[36622]: pgmap v444: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 13 op/s 2024-06-03T01:24:32.785 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:24:32.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:32 smithi094 ceph-mon[32119]: pgmap v444: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 13 op/s 2024-06-03T01:24:32.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:32 smithi179 ceph-mon[37041]: pgmap v444: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 92 KiB/s wr, 13 op/s 2024-06-03T01:24:34.483 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:34 smithi107 ceph-mon[36622]: pgmap v445: 129 pgs: 129 active+clean; 355 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 20 op/s 2024-06-03T01:24:34.615 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-06-03T01:24:34.615 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2367, 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2369, 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 71146618, 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 752716, 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 8, 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.220783501, 2024-06-03T01:24:34.616 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.069799943, 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.078708674, 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.055064443, 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2367, 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.077872401, 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000032899 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.617 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.618 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:34.619 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1205, 2024-06-03T01:24:34.619 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1146, 2024-06-03T01:24:34.619 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 65084640, 2024-06-03T01:24:34.619 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 458544, 2024-06-03T01:24:34.619 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-06-03T01:24:34.619 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-06-03T01:24:34.619 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.154122800, 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.033400263, 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.099029561, 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.026864141, 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1146, 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.038419462, 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000033524 2024-06-03T01:24:34.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1953616, 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 206304 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.621 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 796, 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1043, 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 37947731, 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 822861, 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 8, 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.100479014, 2024-06-03T01:24:34.622 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.025549021, 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.044113238, 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.023032616, 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1040, 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.045563050, 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000043810 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:34.623 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 570368 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-06-03T01:24:34.624 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-06-03T01:24:34.625 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-06-03T01:24:34.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.625 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-06-03T01:24:34.625 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:34.625 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:34.625 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6731, 2024-06-03T01:24:34.625 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.188671381, 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000028030 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.626 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-06-03T01:24:34.627 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:34.628 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 3419, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 1927, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 1492, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 666, 2024-06-03T01:24:34.629 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-06-03T01:24:34.630 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-06-03T01:24:34.630 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-06-03T01:24:34.630 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 11, 2024-06-03T01:24:34.630 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-06-03T01:24:34.630 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-06-03T01:24:34.630 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-06-03T01:24:34.630 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-06-03T01:24:34.631 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:34.631 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:34.631 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-06-03T01:24:34.631 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-06-03T01:24:34.631 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-06-03T01:24:34.631 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-06-03T01:24:34.631 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-06-03T01:24:34.632 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-06-03T01:24:34.632 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-06-03T01:24:34.632 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:34.632 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:34.632 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:34.632 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 590, 2024-06-03T01:24:34.632 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 204782854, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:34.633 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:34.634 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-06-03T01:24:34.635 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-06-03T01:24:34.635 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-06-03T01:24:34.635 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-06-03T01:24:34.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.635 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-06-03T01:24:34.635 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-06-03T01:24:34.635 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-06-03T01:24:34.636 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-06-03T01:24:34.637 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:34.638 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:34.638 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:34.638 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-06-03T01:24:34.638 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-06-03T01:24:34.638 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:34.638 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:34.638 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 31717, 2024-06-03T01:24:34.639 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 3, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 64, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-06-03T01:24:34.640 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 168761271, 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 175311135, 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 175306462, 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 31717 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 215, 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 5154, 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 4939, 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 48, 2024-06-03T01:24:34.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 147, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 99, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 3419, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 6025, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 2606, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 65144, 2024-06-03T01:24:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215420 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-06-03T01:24:34.644 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.645 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-06-03T01:24:34.646 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.648 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.649 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.649 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-06-03T01:24:34.649 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.649 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.649 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.650 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:34.650 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.650 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.650 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.650 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:34.650 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.652 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-06-03T01:24:34.652 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-06-03T01:24:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.653 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.654 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-06-03T01:24:34.654 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.654 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.654 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.654 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-06-03T01:24:34.655 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.655 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.655 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.655 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-06-03T01:24:34.655 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.655 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.656 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.657 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:34.658 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.661 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-06-03T01:24:34.662 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.662 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.662 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.662 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-06-03T01:24:34.662 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.662 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-06-03T01:24:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.664 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.664 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.664 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:34.664 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:34.664 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-06-03T01:24:34.665 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 5, 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 3, 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 3, 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 3579, 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:34.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.667 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-06-03T01:24:34.667 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:34.667 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:34.667 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:34.667 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:34.667 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:34.668 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:34.669 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:34.670 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:34.670 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:34.670 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:34.670 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:34.670 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:34.670 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:34.670 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:34.671 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 249768, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 319, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:34.672 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:34.673 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:34.673 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 181, 2024-06-03T01:24:34.673 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:34.673 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:34.673 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:34.673 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-06-03T01:24:34.673 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 3040578, 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 23180, 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-06-03T01:24:34.674 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-06-03T01:24:34.675 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-06-03T01:24:34.675 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3954, 2024-06-03T01:24:34.675 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 26, 2024-06-03T01:24:34.675 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 4, 2024-06-03T01:24:34.675 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3954, 2024-06-03T01:24:34.675 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-06-03T01:24:34.675 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3954, 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3.545943190, 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000896798 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3954, 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3955 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.676 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3954, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 3954, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 2194, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1757, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-06-03T01:24:34.677 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:34.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 2, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-06-03T01:24:34.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-06-03T01:24:34.680 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-06-03T01:24:34.681 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 117, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 19, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 11, 2024-06-03T01:24:34.682 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 11, 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-06-03T01:24:34.683 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.684 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:34.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 4364, 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 173762528, 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:34.686 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 4364, 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 4364, 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 173762528, 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.687 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:34.688 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:34.689 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:34.689 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:34.689 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3954, 2024-06-03T01:24:34.689 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 171130085, 2024-06-03T01:24:34.689 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3954, 2024-06-03T01:24:34.689 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 171130085, 2024-06-03T01:24:34.689 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:34.690 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.690 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.690 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.690 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.690 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:34.690 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3954, 2024-06-03T01:24:34.691 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3954, 2024-06-03T01:24:34.692 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3954, 2024-06-03T01:24:34.692 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3954, 2024-06-03T01:24:34.692 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:34.692 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.692 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.692 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:34.693 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.694 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.695 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.695 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:34.695 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x565025aac6e0": { 2024-06-03T01:24:34.695 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:34.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:34.695 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-06-03T01:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-06-03T01:24:34.697 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:34.697 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:34.697 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:34.697 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:34.697 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.697 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:34.697 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-06-03T01:24:34.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:34 smithi094 ceph-mon[32119]: pgmap v445: 129 pgs: 129 active+clean; 355 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 20 op/s 2024-06-03T01:24:34.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:34 smithi179 ceph-mon[37041]: pgmap v445: 129 pgs: 129 active+clean; 355 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 145 KiB/s wr, 20 op/s 2024-06-03T01:24:35.412 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-03T01:24:35.412 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-03T01:24:35.412 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-06-03T01:24:35.412 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:35.412 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:35.413 DEBUG:tasks.check_counter:Getting stats from h 2024-06-03T01:24:35.413 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-03T01:24:35.650 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:24:36.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:36 smithi107 ceph-mon[36622]: pgmap v446: 129 pgs: 129 active+clean; 355 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 142 KiB/s wr, 15 op/s 2024-06-03T01:24:36.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:36 smithi094 ceph-mon[32119]: pgmap v446: 129 pgs: 129 active+clean; 355 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 142 KiB/s wr, 15 op/s 2024-06-03T01:24:36.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:36 smithi179 ceph-mon[37041]: pgmap v446: 129 pgs: 129 active+clean; 355 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 142 KiB/s wr, 15 op/s 2024-06-03T01:24:38.079 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 29121, 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 43122, 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 9657652, 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 36914931, 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 19, 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-06-03T01:24:38.080 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 1.872143202, 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.777091172, 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 1.249541115, 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.082841363, 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 43120, 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.839355132, 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000019465 2024-06-03T01:24:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.082 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:38.082 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 58746, 2024-06-03T01:24:38.082 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.010277058, 2024-06-03T01:24:38.082 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000174 2024-06-03T01:24:38.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 5776, 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 4064 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 5627, 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 6185, 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3758770, 2024-06-03T01:24:38.083 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 11637314, 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 9, 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 7, 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.440779886, 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.147678656, 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.154520608, 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.130930189, 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:38.084 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6184, 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.189232555, 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000030600 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1846, 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000268904, 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000145 2024-06-03T01:24:38.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1968448, 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 605712 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 36412, 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 33897, 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 12314148, 2024-06-03T01:24:38.086 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 20234575, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 11, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 2.010301260, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.882524172, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 1.310620762, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.099395633, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 33894, 2024-06-03T01:24:38.087 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.613851624, 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000018110 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 63125, 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.009407248, 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000149 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.088 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 254304 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-06-03T01:24:38.089 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 8873, 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 5.402532350, 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000608873 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:38.090 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 5488, 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.304899163, 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000055557 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 17429, 2024-06-03T01:24:38.091 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 16935, 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 16935, 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 102.560727340, 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.006056139 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 494, 2024-06-03T01:24:38.092 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 3, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 11, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 452, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 50, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 336, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 66, 2024-06-03T01:24:38.093 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 380, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 133, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 151, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 151, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 8, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 41504, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 21667, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 493, 2024-06-03T01:24:38.094 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 54, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 993, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 2720, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 2, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 65, 2024-06-03T01:24:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 7, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 398, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 588, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 202130694, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-06-03T01:24:38.096 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 18, 2024-06-03T01:24:38.097 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 1309, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 11329, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 5193, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 2092, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 1568, 2024-06-03T01:24:38.098 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 16427 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 3, 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 2, 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 56, 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 46, 2024-06-03T01:24:38.099 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 2, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 2370, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 2368, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:38.100 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 4, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-06-03T01:24:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 17882, 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-06-03T01:24:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 17882, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 21, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 21, 2024-06-03T01:24:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 3, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 31848776, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4214, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 15.976375907, 2024-06-03T01:24:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.003791261 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 386, 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 2818, 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 2432, 2024-06-03T01:24:38.105 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 50, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 58, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 8, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 452, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 4421, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 3969, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 149, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 2569, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 2420, 2024-06-03T01:24:38.106 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 62036, 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 223612 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 33512, 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 10, 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 17429, 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 348, 2024-06-03T01:24:38.107 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 6116, 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2328, 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 5.225333679, 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002244559 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 76, 2024-06-03T01:24:38.108 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.083395892, 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001097314 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-06-03T01:24:38.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4781, 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 4.945528855, 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001034413 2024-06-03T01:24:38.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.111 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:38.111 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.111 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.111 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.111 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.114 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-06-03T01:24:38.116 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6, 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.003431240, 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000571873 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 6316, 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.626329629, 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000415821 2024-06-03T01:24:38.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.118 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-06-03T01:24:38.118 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1039, 2024-06-03T01:24:38.118 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 84.844827070, 2024-06-03T01:24:38.118 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.081660083 2024-06-03T01:24:38.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.118 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:38.118 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.120 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-06-03T01:24:38.121 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 19, 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.097760746, 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005145302 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.123 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.124 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2370, 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 4.734120229, 2024-06-03T01:24:38.125 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001997519 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 3, 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 2, 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-06-03T01:24:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 2852, 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 2852, 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 754, 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:38.128 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:38.129 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:38.130 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 335091, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 499, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 88, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 1, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:38.133 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 181, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-06-03T01:24:38.134 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 1423475, 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 12772, 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-06-03T01:24:38.135 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 10423, 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 28216175, 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 28, 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 10423, 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 10423, 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 51.155181932, 2024-06-03T01:24:38.136 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004907913 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 10423, 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 11006 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 10423, 2024-06-03T01:24:38.137 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 385, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 10038, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 0, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 10, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 382, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 4596, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 254, 2024-06-03T01:24:38.138 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 4754, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:38.139 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 0, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 20, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-06-03T01:24:38.140 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-06-03T01:24:38.141 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-06-03T01:24:38.142 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 117, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 19, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 12, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 12, 2024-06-03T01:24:38.143 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 774, 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 6, 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 210 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-06-03T01:24:38.144 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:38.145 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 10, 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 5, 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 4736, 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 2368, 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:38.146 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 71156, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 19301072, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 71156, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:38.147 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 71156, 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 19301072, 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.148 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:38.149 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 10423, 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 28474655, 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 10423, 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 28474655, 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.150 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:38.151 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 10423, 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 10423, 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 10423, 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 10423, 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 2368, 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 258528, 2024-06-03T01:24:38.153 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 2368, 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 382, 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 258528, 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.154 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x563a96650500": { 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 18838, 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 17882, 2024-06-03T01:24:38.155 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 27654472, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 17882, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 4214, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 27635634, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:38.156 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:38.157 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:38.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:38.157 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-06-03T01:24:38.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:38 smithi107 ceph-mon[36622]: pgmap v447: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 165 KiB/s wr, 28 op/s 2024-06-03T01:24:38.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:38 smithi094 ceph-mon[32119]: pgmap v447: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 165 KiB/s wr, 28 op/s 2024-06-03T01:24:38.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:38 smithi179 ceph-mon[37041]: pgmap v447: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 165 KiB/s wr, 28 op/s 2024-06-03T01:24:38.983 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-06-03T01:24:38.983 INFO:tasks.check_counter:Daemon mds.h mds.imported=7 2024-06-03T01:24:38.983 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-06-03T01:24:38.983 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:38.984 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:38.984 DEBUG:tasks.check_counter:Getting stats from k 2024-06-03T01:24:38.984 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-03T01:24:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.b/config 2024-06-03T01:24:40.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:40 smithi094 ceph-mon[32119]: pgmap v448: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-06-03T01:24:40.681 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:40 smithi107 ceph-mon[36622]: pgmap v448: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-06-03T01:24:40.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:40 smithi179 ceph-mon[37041]: pgmap v448: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-06-03T01:24:40.943 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-06-03T01:24:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 928, 2024-06-03T01:24:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 929, 2024-06-03T01:24:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 8628759, 2024-06-03T01:24:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 294850, 2024-06-03T01:24:40.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 8, 2024-06-03T01:24:40.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-06-03T01:24:40.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.082976345, 2024-06-03T01:24:40.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.026775208, 2024-06-03T01:24:40.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.025252009, 2024-06-03T01:24:40.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.023106771, 2024-06-03T01:24:40.945 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 927, 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.045258206, 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000048822 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1424, 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1366, 2024-06-03T01:24:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 10815326, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 530211, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.152427989, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.043595882, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.073243110, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.037378474, 2024-06-03T01:24:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1366, 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.047262690, 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000034599 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1954816, 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 209344 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1487, 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1738, 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 4661340, 2024-06-03T01:24:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 1044918, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.159280563, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.054887714, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.040131709, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.048099793, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1735, 2024-06-03T01:24:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.078683453, 2024-06-03T01:24:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000045350 2024-06-03T01:24:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 572688 2024-06-03T01:24:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-06-03T01:24:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-06-03T01:24:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-06-03T01:24:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-06-03T01:24:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 5954, 2024-06-03T01:24:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.145982208, 2024-06-03T01:24:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000024518 2024-06-03T01:24:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-06-03T01:24:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-06-03T01:24:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-06-03T01:24:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-06-03T01:24:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-06-03T01:24:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-06-03T01:24:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-06-03T01:24:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-06-03T01:24:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-06-03T01:24:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 2852, 2024-06-03T01:24:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 140, 2024-06-03T01:24:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 2712, 2024-06-03T01:24:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-06-03T01:24:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 2651, 2024-06-03T01:24:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-06-03T01:24:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-06-03T01:24:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-06-03T01:24:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 5, 2024-06-03T01:24:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-06-03T01:24:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-06-03T01:24:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-06-03T01:24:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-06-03T01:24:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-06-03T01:24:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-06-03T01:24:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-06-03T01:24:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-06-03T01:24:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-06-03T01:24:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-06-03T01:24:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-06-03T01:24:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-06-03T01:24:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-06-03T01:24:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-06-03T01:24:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-06-03T01:24:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T01:24:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-06-03T01:24:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-06-03T01:24:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-06-03T01:24:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-06-03T01:24:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-06-03T01:24:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-06-03T01:24:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-06-03T01:24:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-06-03T01:24:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-06-03T01:24:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-06-03T01:24:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-06-03T01:24:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-06-03T01:24:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-06-03T01:24:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-06-03T01:24:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-06-03T01:24:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-06-03T01:24:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-06-03T01:24:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-06-03T01:24:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-06-03T01:24:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-06-03T01:24:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-06-03T01:24:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-06-03T01:24:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-06-03T01:24:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-06-03T01:24:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-06-03T01:24:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 13774, 2024-06-03T01:24:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-06-03T01:24:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-06-03T01:24:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-06-03T01:24:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-06-03T01:24:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-06-03T01:24:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 17, 2024-06-03T01:24:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 3, 2024-06-03T01:24:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-06-03T01:24:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-06-03T01:24:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-06-03T01:24:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 25383751, 2024-06-03T01:24:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 25380520, 2024-06-03T01:24:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-06-03T01:24:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 13774 2024-06-03T01:24:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-06-03T01:24:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 478, 2024-06-03T01:24:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 2862, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 2384, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 52, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 67, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 15, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 2865, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 3052, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 187, 2024-06-03T01:24:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-06-03T01:24:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-06-03T01:24:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-06-03T01:24:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 55372, 2024-06-03T01:24:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215420 2024-06-03T01:24:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-06-03T01:24:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-06-03T01:24:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-06-03T01:24:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-06-03T01:24:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-06-03T01:24:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-06-03T01:24:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-06-03T01:24:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-06-03T01:24:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-06-03T01:24:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-06-03T01:24:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-06-03T01:24:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-06-03T01:24:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-06-03T01:24:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-06-03T01:24:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-06-03T01:24:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-06-03T01:24:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-06-03T01:24:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-06-03T01:24:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-06-03T01:24:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-06-03T01:24:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-06-03T01:24:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-06-03T01:24:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-06-03T01:24:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2024-06-03T01:24:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2024-06-03T01:24:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-06-03T01:24:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-06-03T01:24:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-06-03T01:24:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-06-03T01:24:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 2672, 2024-06-03T01:24:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-06-03T01:24:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-06-03T01:24:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 225757, 2024-06-03T01:24:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 548, 2024-06-03T01:24:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-06-03T01:24:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-06-03T01:24:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 181, 2024-06-03T01:24:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-06-03T01:24:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 3229191, 2024-06-03T01:24:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 22647, 2024-06-03T01:24:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-06-03T01:24:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3421, 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-06-03T01:24:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3421, 2024-06-03T01:24:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-06-03T01:24:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3421, 2024-06-03T01:24:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3.092950360, 2024-06-03T01:24:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000904107 2024-06-03T01:24:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3421, 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3422 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3421, 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 3421, 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-06-03T01:24:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-06-03T01:24:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-06-03T01:24:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1955, 2024-06-03T01:24:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-06-03T01:24:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1464, 2024-06-03T01:24:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-06-03T01:24:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-06-03T01:24:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-06-03T01:24:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-06-03T01:24:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-06-03T01:24:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-06-03T01:24:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-06-03T01:24:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-06-03T01:24:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-06-03T01:24:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-06-03T01:24:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-06-03T01:24:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-06-03T01:24:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-06-03T01:24:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-06-03T01:24:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-06-03T01:24:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-06-03T01:24:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-06-03T01:24:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-06-03T01:24:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-06-03T01:24:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-06-03T01:24:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-06-03T01:24:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-06-03T01:24:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-06-03T01:24:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-06-03T01:24:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-06-03T01:24:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-06-03T01:24:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-06-03T01:24:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-06-03T01:24:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-06-03T01:24:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-06-03T01:24:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-06-03T01:24:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 117, 2024-06-03T01:24:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-06-03T01:24:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 19, 2024-06-03T01:24:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 7, 2024-06-03T01:24:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 8, 2024-06-03T01:24:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 1, 2024-06-03T01:24:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-06-03T01:24:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-06-03T01:24:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-06-03T01:24:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-06-03T01:24:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-06-03T01:24:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T01:24:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-06-03T01:24:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-06-03T01:24:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-06-03T01:24:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-06-03T01:24:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3835, 2024-06-03T01:24:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 23736360, 2024-06-03T01:24:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3835, 2024-06-03T01:24:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3835, 2024-06-03T01:24:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 23736360, 2024-06-03T01:24:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-06-03T01:24:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3421, 2024-06-03T01:24:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 21189879, 2024-06-03T01:24:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3421, 2024-06-03T01:24:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 21189879, 2024-06-03T01:24:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-06-03T01:24:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3421, 2024-06-03T01:24:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3421, 2024-06-03T01:24:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3421, 2024-06-03T01:24:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3421, 2024-06-03T01:24:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-06-03T01:24:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-06-03T01:24:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x556df2fc06e0": { 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-06-03T01:24:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-06-03T01:24:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-06-03T01:24:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-06-03T01:24:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-06-03T01:24:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-06-03T01:24:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-06-03T01:24:41.034 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-06-03T01:24:41.784 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-06-03T01:24:41.784 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-06-03T01:24:41.784 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-06-03T01:24:41.784 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:41.784 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:41.784 DEBUG:tasks.check_counter:Getting stats from c 2024-06-03T01:24:41.785 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-03T01:24:42.035 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:24:42.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:42 smithi107 ceph-mon[36622]: pgmap v449: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-06-03T01:24:42.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:42 smithi094 ceph-mon[32119]: pgmap v449: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-06-03T01:24:42.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:42 smithi179 ceph-mon[37041]: pgmap v449: 129 pgs: 129 active+clean; 353 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-06-03T01:24:43.773 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:43 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:43.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:43 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:43.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:43 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:24:43.899 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-06-03T01:24:43.899 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:43.899 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 52823, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 62507, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 18019814, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 42507398, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 12, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 2.809623250, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 1.148382408, 2024-06-03T01:24:43.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 2.005024358, 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.098297300, 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 62503, 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.038255932, 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000016611 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:43.901 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 99933, 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.015668414, 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000156 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:43.902 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 5477, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 5000, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 3610110, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 10822957, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 9, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.411372972, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.143155379, 2024-06-03T01:24:43.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.168224555, 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.108276676, 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4999, 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.147364925, 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000029478 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:43.904 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1759, 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000294326, 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000167 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1980832, 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 211376 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2822, 2024-06-03T01:24:43.905 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 3028, 2024-06-03T01:24:43.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1358993, 2024-06-03T01:24:43.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 6334702, 2024-06-03T01:24:43.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 16, 2024-06-03T01:24:43.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2024-06-03T01:24:43.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.211873461, 2024-06-03T01:24:43.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.072741226, 2024-06-03T01:24:43.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.066309808, 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.063901437, 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3028, 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.121623709, 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000040166 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 782, 2024-06-03T01:24:43.907 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000148777, 2024-06-03T01:24:43.908 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000190 2024-06-03T01:24:43.908 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 4992, 2024-06-03T01:24:43.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 578560 2024-06-03T01:24:43.908 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.908 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-06-03T01:24:43.908 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7794, 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 4.132031862, 2024-06-03T01:24:43.909 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000530155 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5232, 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.270202806, 2024-06-03T01:24:43.910 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000051644 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 13074, 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 12587, 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 12587, 2024-06-03T01:24:43.911 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 16.798820547, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001334616 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 487, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-06-03T01:24:43.912 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 675, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 147, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 512, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 16, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 465, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 199, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 207, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 207, 2024-06-03T01:24:43.913 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 8, 2024-06-03T01:24:43.914 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 31953, 2024-06-03T01:24:43.914 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 16235, 2024-06-03T01:24:43.914 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 486, 2024-06-03T01:24:43.914 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 25, 2024-06-03T01:24:43.914 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:43.914 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:43.914 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 738, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 5076, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 5, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 172, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 8, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 684, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:43.915 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 590, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 204782854, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:43.916 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 1186, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 9767, 2024-06-03T01:24:43.917 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 3762, 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 1393, 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 785, 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 12910 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.918 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 3, 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 2, 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 28, 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 17, 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:43.919 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:43.920 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 2215, 2024-06-03T01:24:43.920 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 2215, 2024-06-03T01:24:43.920 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:43.920 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-06-03T01:24:43.920 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:43.920 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:43.920 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 5, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:43.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 13774, 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-06-03T01:24:43.922 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 13774, 2024-06-03T01:24:43.923 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-06-03T01:24:43.923 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-06-03T01:24:43.923 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 17, 2024-06-03T01:24:43.923 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-06-03T01:24:43.923 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-06-03T01:24:43.923 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 17, 2024-06-03T01:24:43.923 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 3, 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 25383751, 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3689, 2024-06-03T01:24:43.924 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 13.966717485, 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003786044 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 506, 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 2874, 2024-06-03T01:24:43.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 2368, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 62, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 80, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 18, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 675, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 3225, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 2550, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 207, 2024-06-03T01:24:43.926 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 2538, 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 2331, 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 63256, 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223612 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 26290, 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 239, 2024-06-03T01:24:43.927 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 13074, 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 442, 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 5314, 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2226, 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 5.444981459, 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002446083 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.928 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000220687, 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000220687 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.929 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.930 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-06-03T01:24:43.930 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.930 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.930 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.930 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.930 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-06-03T01:24:43.930 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2875, 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.451545113, 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001200537 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.931 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.932 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:43.933 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.934 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.934 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.934 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.934 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-06-03T01:24:43.935 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.935 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.935 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.935 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.935 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.936 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-06-03T01:24:43.937 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.938 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.938 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.938 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.938 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-06-03T01:24:43.938 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4204, 2024-06-03T01:24:43.938 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.027186117, 2024-06-03T01:24:43.938 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000482204 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 738, 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.807896285, 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002449723 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:43.939 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.940 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.941 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-06-03T01:24:43.942 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.943 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 328, 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.124547204, 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000379717 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.944 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.945 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2215, 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.942443682, 2024-06-03T01:24:43.946 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001779884 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.947 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 2, 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-06-03T01:24:43.948 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 2087, 2024-06-03T01:24:43.949 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 2087, 2024-06-03T01:24:43.949 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 761, 2024-06-03T01:24:43.949 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:43.949 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.949 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-06-03T01:24:43.949 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:43.949 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:43.950 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:43.950 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:43.950 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:43.950 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:43.950 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:43.950 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:43.951 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:43.951 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:43.951 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:43.951 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:43.951 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:43.951 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:43.951 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:43.952 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:43.953 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:43.953 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:43.953 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:43.953 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:43.953 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:43.953 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:43.953 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 476545, 2024-06-03T01:24:43.954 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 666, 2024-06-03T01:24:43.955 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:43.955 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:43.955 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-06-03T01:24:43.955 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-06-03T01:24:43.955 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:43.955 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:43.955 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:43.956 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:43.956 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:43.956 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 181, 2024-06-03T01:24:43.956 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:43.956 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:43.956 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 1940625, 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 17159, 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-06-03T01:24:43.957 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 9537, 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 21744122, 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 28, 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 9537, 2024-06-03T01:24:43.958 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 9537, 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 44.993957242, 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004717831 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 9537, 2024-06-03T01:24:43.959 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 10137 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 9537, 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 359, 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 9178, 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-06-03T01:24:43.960 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 359, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 4048, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 245, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-06-03T01:24:43.961 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 4447, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:43.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-06-03T01:24:43.963 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-06-03T01:24:43.964 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 117, 2024-06-03T01:24:43.965 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 762, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-06-03T01:24:43.966 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 246 2024-06-03T01:24:43.967 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.967 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-06-03T01:24:43.967 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-06-03T01:24:43.967 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:43.967 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-06-03T01:24:43.967 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-06-03T01:24:43.967 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 10, 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 5, 2024-06-03T01:24:43.968 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 4430, 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 2215, 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:43.969 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 61118, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 17462869, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 61118, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 61118, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 17462869, 2024-06-03T01:24:43.970 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:43.971 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 9537, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 21986421, 2024-06-03T01:24:43.972 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 9537, 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 21986421, 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.973 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:43.974 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:43.974 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-06-03T01:24:43.974 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:43.974 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:43.974 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 9537, 2024-06-03T01:24:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 9537, 2024-06-03T01:24:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 9537, 2024-06-03T01:24:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 9537, 2024-06-03T01:24:43.978 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:43.979 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 2215, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 242299, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 2215, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:43.980 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 359, 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 242299, 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:43.981 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x555561fe4500": { 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 13774, 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 21189447, 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 13774, 2024-06-03T01:24:43.982 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3689, 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 21189447, 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:43.983 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.984 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:43.984 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-06-03T01:24:44.769 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:44 smithi179 ceph-mon[37041]: pgmap v450: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 530 KiB/s wr, 31 op/s 2024-06-03T01:24:44.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:44 smithi107 ceph-mon[36622]: pgmap v450: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 530 KiB/s wr, 31 op/s 2024-06-03T01:24:44.794 INFO:tasks.check_counter:Daemon mds.c mds.exported=5 2024-06-03T01:24:44.794 INFO:tasks.check_counter:Daemon mds.c mds.imported=8 2024-06-03T01:24:44.795 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-06-03T01:24:44.795 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:44.795 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:44.795 DEBUG:tasks.check_counter:Getting stats from f 2024-06-03T01:24:44.795 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-03T01:24:44.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:44 smithi094 ceph-mon[32119]: pgmap v450: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 530 KiB/s wr, 31 op/s 2024-06-03T01:24:45.043 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:24:46.764 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:46 smithi179 ceph-mon[37041]: pgmap v451: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 476 KiB/s wr, 24 op/s 2024-06-03T01:24:46.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:46 smithi107 ceph-mon[36622]: pgmap v451: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 476 KiB/s wr, 24 op/s 2024-06-03T01:24:46.801 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-06-03T01:24:46.801 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:46.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 17201, 2024-06-03T01:24:46.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 21297, 2024-06-03T01:24:46.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 5966595, 2024-06-03T01:24:46.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 25008406, 2024-06-03T01:24:46.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 11, 2024-06-03T01:24:46.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-06-03T01:24:46.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 1.057781885, 2024-06-03T01:24:46.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.438601475, 2024-06-03T01:24:46.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.713418277, 2024-06-03T01:24:46.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.050941775, 2024-06-03T01:24:46.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:46.803 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 21296, 2024-06-03T01:24:46.803 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.452326608, 2024-06-03T01:24:46.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000021239 2024-06-03T01:24:46.804 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:46.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 30440, 2024-06-03T01:24:46.804 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.005678245, 2024-06-03T01:24:46.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000186 2024-06-03T01:24:46.804 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.805 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-03T01:24:46.805 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T01:24:46.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.805 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:46.805 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 18780, 2024-06-03T01:24:46.805 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 13582, 2024-06-03T01:24:46.805 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 7722331, 2024-06-03T01:24:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 4803910, 2024-06-03T01:24:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 14, 2024-06-03T01:24:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-06-03T01:24:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.933131100, 2024-06-03T01:24:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.361879616, 2024-06-03T01:24:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.728175083, 2024-06-03T01:24:46.806 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.021399962, 2024-06-03T01:24:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 13578, 2024-06-03T01:24:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.281817753, 2024-06-03T01:24:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000020755 2024-06-03T01:24:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:46.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 30684, 2024-06-03T01:24:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.004867300, 2024-06-03T01:24:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000158 2024-06-03T01:24:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3136, 2024-06-03T01:24:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 263168 2024-06-03T01:24:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.808 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 3567, 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 3247, 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 3152608, 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 10215128, 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 14, 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.292455506, 2024-06-03T01:24:46.809 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.115026271, 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.140579849, 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.056901724, 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3246, 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.130115716, 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000040084 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.810 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:46.811 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1942, 2024-06-03T01:24:46.811 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000313118, 2024-06-03T01:24:46.811 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000161 2024-06-03T01:24:46.811 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.811 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1970816, 2024-06-03T01:24:46.811 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 611504 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:46.812 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:46.813 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3804, 2024-06-03T01:24:46.813 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.602853286, 2024-06-03T01:24:46.813 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000421359 2024-06-03T01:24:46.813 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.813 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.813 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:46.814 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:46.814 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:46.814 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2202, 2024-06-03T01:24:46.814 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.104525270, 2024-06-03T01:24:46.814 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000047468 2024-06-03T01:24:46.814 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.814 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.815 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-06-03T01:24:46.815 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 9713, 2024-06-03T01:24:46.815 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 9579, 2024-06-03T01:24:46.815 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-06-03T01:24:46.815 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 9579, 2024-06-03T01:24:46.815 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 20.999254992, 2024-06-03T01:24:46.815 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002192217 2024-06-03T01:24:46.816 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.816 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-06-03T01:24:46.816 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 134, 2024-06-03T01:24:46.816 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 7, 2024-06-03T01:24:46.816 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:46.816 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-06-03T01:24:46.816 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-06-03T01:24:46.817 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-06-03T01:24:46.817 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 381, 2024-06-03T01:24:46.817 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 49, 2024-06-03T01:24:46.817 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 318, 2024-06-03T01:24:46.817 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 14, 2024-06-03T01:24:46.817 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 312, 2024-06-03T01:24:46.817 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 232, 2024-06-03T01:24:46.818 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 139, 2024-06-03T01:24:46.818 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 139, 2024-06-03T01:24:46.818 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 7, 2024-06-03T01:24:46.818 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 14855, 2024-06-03T01:24:46.818 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 8567, 2024-06-03T01:24:46.818 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 126, 2024-06-03T01:24:46.818 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 19, 2024-06-03T01:24:46.819 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 1, 2024-06-03T01:24:46.819 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:46.819 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-06-03T01:24:46.819 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 3267, 2024-06-03T01:24:46.819 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-06-03T01:24:46.819 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 7, 2024-06-03T01:24:46.820 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 184, 2024-06-03T01:24:46.820 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 9, 2024-06-03T01:24:46.820 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 433, 2024-06-03T01:24:46.820 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:46.820 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:46.820 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 1, 2024-06-03T01:24:46.820 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 590, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 204782854, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:46.821 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:46.822 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:46.822 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:46.822 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-06-03T01:24:46.822 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 1213, 2024-06-03T01:24:46.822 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 4643, 2024-06-03T01:24:46.822 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:46.822 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:46.823 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:46.823 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 3417, 2024-06-03T01:24:46.823 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 1088, 2024-06-03T01:24:46.823 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 631, 2024-06-03T01:24:46.823 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 9134 2024-06-03T01:24:46.823 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.823 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 3, 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 2, 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 22, 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 17, 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:46.824 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:46.825 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 737, 2024-06-03T01:24:46.825 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 737, 2024-06-03T01:24:46.825 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:46.825 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-06-03T01:24:46.825 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:46.825 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:46.825 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:46.826 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-06-03T01:24:46.826 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-06-03T01:24:46.826 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:46.826 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:46.826 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:46.826 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 14, 2024-06-03T01:24:46.826 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-06-03T01:24:46.827 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:46.827 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:46.827 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-06-03T01:24:46.827 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.827 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-06-03T01:24:46.827 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-06-03T01:24:46.827 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 10390, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 10390, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 13, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-06-03T01:24:46.828 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-06-03T01:24:46.829 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 13, 2024-06-03T01:24:46.829 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-06-03T01:24:46.829 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-06-03T01:24:46.829 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-06-03T01:24:46.829 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-06-03T01:24:46.829 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 17626857, 2024-06-03T01:24:46.829 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-06-03T01:24:46.830 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-06-03T01:24:46.830 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1715, 2024-06-03T01:24:46.830 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 5.800127754, 2024-06-03T01:24:46.830 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003381998 2024-06-03T01:24:46.830 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.830 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-06-03T01:24:46.830 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.831 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-06-03T01:24:46.831 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 351, 2024-06-03T01:24:46.831 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 1280, 2024-06-03T01:24:46.831 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 929, 2024-06-03T01:24:46.831 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 54, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 71, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 17, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 381, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 2110, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 1729, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 139, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1102, 2024-06-03T01:24:46.832 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 963, 2024-06-03T01:24:46.833 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 61000, 2024-06-03T01:24:46.833 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223612 2024-06-03T01:24:46.833 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.833 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-06-03T01:24:46.833 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 13646, 2024-06-03T01:24:46.833 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 269, 2024-06-03T01:24:46.833 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 9713, 2024-06-03T01:24:46.834 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 141, 2024-06-03T01:24:46.834 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 1618, 2024-06-03T01:24:46.834 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-06-03T01:24:46.834 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 739, 2024-06-03T01:24:46.834 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 8.523800963, 2024-06-03T01:24:46.834 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.011534236 2024-06-03T01:24:46.834 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 11, 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.016667877, 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001515261 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.835 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.836 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.836 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.836 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-06-03T01:24:46.836 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.836 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.836 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.836 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.837 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-06-03T01:24:46.837 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1176, 2024-06-03T01:24:46.837 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.326386886, 2024-06-03T01:24:46.837 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000277539 2024-06-03T01:24:46.837 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.837 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:46.837 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.838 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.838 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.838 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.838 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-06-03T01:24:46.838 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-06-03T01:24:46.838 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000110004, 2024-06-03T01:24:46.838 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000110004 2024-06-03T01:24:46.839 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.839 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-06-03T01:24:46.839 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.839 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.839 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.839 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.839 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:46.840 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.840 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.840 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.840 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.840 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:46.840 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.840 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.841 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.841 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.841 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-06-03T01:24:46.842 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.842 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.842 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.842 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.842 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-06-03T01:24:46.842 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.842 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.843 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.843 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.843 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-06-03T01:24:46.843 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.843 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.843 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.844 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.844 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-06-03T01:24:46.844 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.844 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.845 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.845 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.846 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-06-03T01:24:46.846 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.846 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.846 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.846 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.846 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-06-03T01:24:46.847 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 6915, 2024-06-03T01:24:46.847 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.948018425, 2024-06-03T01:24:46.847 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000426322 2024-06-03T01:24:46.847 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.847 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-06-03T01:24:46.847 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.847 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.848 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.848 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.848 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:46.848 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.848 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.848 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.849 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.849 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:46.849 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.849 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.849 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.849 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.849 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-06-03T01:24:46.850 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.850 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.850 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.850 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.850 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:46.850 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.850 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.851 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.851 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.851 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:46.851 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.851 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.851 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.851 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.852 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-06-03T01:24:46.852 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.852 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.852 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.852 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.852 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:46.852 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.853 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.854 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-06-03T01:24:46.854 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.854 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.854 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.854 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.854 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-06-03T01:24:46.854 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.855 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.855 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.855 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.855 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-06-03T01:24:46.855 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.855 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.855 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.856 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.856 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-06-03T01:24:46.856 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 737, 2024-06-03T01:24:46.856 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 9.184270837, 2024-06-03T01:24:46.856 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.012461697 2024-06-03T01:24:46.856 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.856 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:46.857 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:46.857 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:46.857 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.857 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.857 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.857 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.857 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.858 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-06-03T01:24:46.858 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2024-06-03T01:24:46.858 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 4, 2024-06-03T01:24:46.858 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-06-03T01:24:46.858 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 2, 2024-06-03T01:24:46.858 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 2114, 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1057, 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 840, 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:46.859 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:46.860 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:46.860 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:46.860 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:46.860 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:46.860 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:46.860 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:46.860 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:46.861 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:46.861 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:46.861 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:46.861 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:46.861 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:46.861 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:46.861 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:46.862 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:46.862 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:46.862 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:46.862 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:46.862 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:46.862 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:46.862 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:46.863 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:46.864 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-06-03T01:24:46.864 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:46.864 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:46.864 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:46.864 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:46.864 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 476782, 2024-06-03T01:24:46.864 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 510, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 264, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 3, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:46.865 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:46.866 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:46.866 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 181, 2024-06-03T01:24:46.866 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:46.866 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:46.866 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:46.866 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-06-03T01:24:46.867 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 1326187, 2024-06-03T01:24:46.867 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 11565, 2024-06-03T01:24:46.867 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:46.867 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-06-03T01:24:46.867 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:46.867 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-06-03T01:24:46.867 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 4249, 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 13667695, 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 28, 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 4249, 2024-06-03T01:24:46.868 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4249, 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 20.486526176, 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004821493 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4249, 2024-06-03T01:24:46.869 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 4740 2024-06-03T01:24:46.870 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.870 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 4249, 2024-06-03T01:24:46.870 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 247, 2024-06-03T01:24:46.870 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 4002, 2024-06-03T01:24:46.870 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-06-03T01:24:46.870 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-06-03T01:24:46.870 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-06-03T01:24:46.871 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-06-03T01:24:46.871 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 234, 2024-06-03T01:24:46.871 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 1949, 2024-06-03T01:24:46.871 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 180, 2024-06-03T01:24:46.871 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-06-03T01:24:46.871 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-06-03T01:24:46.871 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-06-03T01:24:46.872 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-06-03T01:24:46.872 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 1489, 2024-06-03T01:24:46.872 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-06-03T01:24:46.872 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:46.872 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:46.872 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 1, 2024-06-03T01:24:46.872 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-06-03T01:24:46.873 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:46.873 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:46.873 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:46.873 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-06-03T01:24:46.873 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-06-03T01:24:46.873 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-06-03T01:24:46.873 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:46.874 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-06-03T01:24:46.874 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:46.874 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-06-03T01:24:46.874 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-06-03T01:24:46.874 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-06-03T01:24:46.874 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-06-03T01:24:46.875 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 117, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-06-03T01:24:46.876 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-06-03T01:24:46.877 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-06-03T01:24:46.877 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-06-03T01:24:46.877 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-06-03T01:24:46.877 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-06-03T01:24:46.877 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 681, 2024-06-03T01:24:46.877 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 20, 2024-06-03T01:24:46.877 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 156 2024-06-03T01:24:46.878 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.878 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-06-03T01:24:46.878 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-06-03T01:24:46.878 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:46.878 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-06-03T01:24:46.878 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-06-03T01:24:46.878 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.879 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-06-03T01:24:46.879 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-06-03T01:24:46.879 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.879 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-06-03T01:24:46.879 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:46.879 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 10, 2024-06-03T01:24:46.879 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-06-03T01:24:46.880 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 5, 2024-06-03T01:24:46.880 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 1474, 2024-06-03T01:24:46.880 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 737, 2024-06-03T01:24:46.880 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:46.880 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.880 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:46.881 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:46.881 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:46.881 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:46.881 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 39544, 2024-06-03T01:24:46.881 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 13256666, 2024-06-03T01:24:46.881 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:46.881 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 39544, 2024-06-03T01:24:46.882 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:46.882 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:46.882 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 39544, 2024-06-03T01:24:46.882 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 13256666, 2024-06-03T01:24:46.882 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:46.882 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.882 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.883 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.883 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.883 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.883 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:46.883 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:46.883 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:46.883 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:46.884 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:46.884 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:46.884 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:46.884 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:46.884 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 4249, 2024-06-03T01:24:46.884 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 13748502, 2024-06-03T01:24:46.885 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4249, 2024-06-03T01:24:46.885 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 13748502, 2024-06-03T01:24:46.885 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:46.885 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.885 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.885 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.886 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.886 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.886 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:46.886 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:46.886 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-06-03T01:24:46.886 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:46.887 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:46.887 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:46.887 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:46.887 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:46.887 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 4249, 2024-06-03T01:24:46.887 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 4249, 2024-06-03T01:24:46.887 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4249, 2024-06-03T01:24:46.888 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 4249, 2024-06-03T01:24:46.888 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:46.888 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.888 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.888 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.888 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.889 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.889 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:46.889 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:46.889 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:46.889 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:46.889 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 737, 2024-06-03T01:24:46.889 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 81093, 2024-06-03T01:24:46.890 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:46.890 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 737, 2024-06-03T01:24:46.890 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:46.890 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:46.890 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 234, 2024-06-03T01:24:46.890 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 81093, 2024-06-03T01:24:46.890 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:46.891 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.891 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.891 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.891 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.891 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:46.891 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55abfef1c500": { 2024-06-03T01:24:46.891 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:46.892 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:46.892 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:46.892 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 10390, 2024-06-03T01:24:46.892 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 13432553, 2024-06-03T01:24:46.892 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:46.892 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 10390, 2024-06-03T01:24:46.893 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:46.893 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:46.893 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 1715, 2024-06-03T01:24:46.893 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 13432553, 2024-06-03T01:24:46.893 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:46.893 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:46.893 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:46.894 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:46.894 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.894 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:46.894 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-06-03T01:24:46.895 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:46 smithi094 ceph-mon[32119]: pgmap v451: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 476 KiB/s wr, 24 op/s 2024-06-03T01:24:47.764 INFO:tasks.check_counter:Daemon mds.f mds.exported=7 2024-06-03T01:24:47.765 INFO:tasks.check_counter:Daemon mds.f mds.imported=9 2024-06-03T01:24:47.765 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-06-03T01:24:47.765 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:47.765 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:47.765 DEBUG:tasks.check_counter:Getting stats from i 2024-06-03T01:24:47.765 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-03T01:24:48.003 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:24:48.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:48 smithi107 ceph-mon[36622]: pgmap v452: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 524 KiB/s wr, 34 op/s 2024-06-03T01:24:48.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:48 smithi094 ceph-mon[32119]: pgmap v452: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 524 KiB/s wr, 34 op/s 2024-06-03T01:24:48.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:48 smithi179 ceph-mon[37041]: pgmap v452: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 524 KiB/s wr, 34 op/s 2024-06-03T01:24:49.711 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-06-03T01:24:49.712 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:49.712 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 7018, 2024-06-03T01:24:49.712 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 7444, 2024-06-03T01:24:49.712 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 2634409, 2024-06-03T01:24:49.712 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 16274354, 2024-06-03T01:24:49.712 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 10, 2024-06-03T01:24:49.712 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-06-03T01:24:49.713 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.514253322, 2024-06-03T01:24:49.713 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.171320381, 2024-06-03T01:24:49.713 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.137501514, 2024-06-03T01:24:49.713 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.175255701, 2024-06-03T01:24:49.713 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:49.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7445, 2024-06-03T01:24:49.713 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.230919864, 2024-06-03T01:24:49.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000031016 2024-06-03T01:24:49.714 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.714 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:49.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1337, 2024-06-03T01:24:49.714 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000254495, 2024-06-03T01:24:49.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000190 2024-06-03T01:24:49.714 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 448 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 83265, 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 109039, 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 25626826, 2024-06-03T01:24:49.715 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 72768350, 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 14, 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 9, 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 4.477625263, 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 1.875924161, 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 2.917556163, 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.209846450, 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:49.716 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 109034, 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.765227543, 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000016189 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 162180, 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.023896178, 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000147 2024-06-03T01:24:49.717 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 928, 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 178448 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 3475, 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 3360, 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 3024990, 2024-06-03T01:24:49.718 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 5975891, 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 14, 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.267875111, 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.091403619, 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.116435443, 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.067120322, 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:49.719 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3359, 2024-06-03T01:24:49.720 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.153706967, 2024-06-03T01:24:49.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000045759 2024-06-03T01:24:49.720 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.720 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:49.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 836, 2024-06-03T01:24:49.720 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000258277, 2024-06-03T01:24:49.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000308 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1986976, 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 613088 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-06-03T01:24:49.721 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.722 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-06-03T01:24:49.722 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:49.722 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:49.722 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 13786, 2024-06-03T01:24:49.722 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 6.396592188, 2024-06-03T01:24:49.722 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000463991 2024-06-03T01:24:49.722 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 9388, 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.529136314, 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000056363 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.723 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 23451, 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 22971, 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 22971, 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 40.616995905, 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001768185 2024-06-03T01:24:49.724 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.725 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-06-03T01:24:49.725 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 480, 2024-06-03T01:24:49.725 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-06-03T01:24:49.725 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:49.725 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-06-03T01:24:49.725 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-06-03T01:24:49.725 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-06-03T01:24:49.726 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 810, 2024-06-03T01:24:49.726 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 359, 2024-06-03T01:24:49.726 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 428, 2024-06-03T01:24:49.726 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 23, 2024-06-03T01:24:49.726 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 297, 2024-06-03T01:24:49.726 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 160, 2024-06-03T01:24:49.726 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 87, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 87, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 8, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 51276, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 21234, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 478, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 7, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:49.727 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:49.728 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 271, 2024-06-03T01:24:49.728 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 4619, 2024-06-03T01:24:49.728 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-06-03T01:24:49.728 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 3, 2024-06-03T01:24:49.728 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 153, 2024-06-03T01:24:49.728 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 5, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 859, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 590, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 211363700, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-06-03T01:24:49.729 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:49.730 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:49.730 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:49.730 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:49.730 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:49.730 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:49.730 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:49.730 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 4, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 587, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 14234, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 4803, 2024-06-03T01:24:49.731 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 2737, 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 531, 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 22341 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 3, 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 2, 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 15, 2024-06-03T01:24:49.732 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 136, 2024-06-03T01:24:49.733 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-06-03T01:24:49.733 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:49.733 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:49.733 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 4112, 2024-06-03T01:24:49.733 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 4112, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-06-03T01:24:49.734 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 4, 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-06-03T01:24:49.735 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 21339, 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 21339, 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-06-03T01:24:49.736 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 27, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 27, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 3, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-06-03T01:24:49.737 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 38841049, 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 6684, 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 17.234257930, 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002578434 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.738 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 632, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 4858, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 4226, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 51, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 63, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 12, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 810, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 6898, 2024-06-03T01:24:49.739 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 6088, 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 87, 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 4439, 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 4352, 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 63244, 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223612 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-06-03T01:24:49.740 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 48904, 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 8, 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 23451, 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 519, 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 8848, 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4110, 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 10.470745102, 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002547626 2024-06-03T01:24:49.741 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 181, 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.125783726, 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000694937 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.742 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-06-03T01:24:49.743 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5219, 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 8.751052655, 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001676768 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.744 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.745 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:49.746 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.747 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.747 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.747 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.747 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-06-03T01:24:49.747 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.747 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-06-03T01:24:49.748 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.749 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001255883, 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000627941 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 8998, 2024-06-03T01:24:49.750 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.921740698, 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000435845 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 332, 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.011822175, 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003047657 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.751 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.752 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.753 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.754 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 17, 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.052223434, 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003071966 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.755 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.756 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.757 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-06-03T01:24:49.758 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.758 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.758 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.758 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.758 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-06-03T01:24:49.758 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4112, 2024-06-03T01:24:49.758 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 16.282372232, 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003959720 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.759 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 3, 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-06-03T01:24:49.760 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 3503, 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 3503, 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 767, 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:49.761 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:49.762 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:49.762 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:49.762 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:49.762 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:49.762 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:49.762 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:49.762 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:49.763 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:49.764 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:49.764 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:49.764 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:49.764 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:49.764 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:49.764 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:49.764 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:49.765 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-06-03T01:24:49.766 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:49.766 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:49.766 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:49.766 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:49.766 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 355230, 2024-06-03T01:24:49.766 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 801, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:49.767 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:49.768 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:49.768 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 181, 2024-06-03T01:24:49.768 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:49.768 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:49.768 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:49.768 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-06-03T01:24:49.768 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 2288274, 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 20633, 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-06-03T01:24:49.769 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 16662, 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 35525455, 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 16662, 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 16662, 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 57.311671179, 2024-06-03T01:24:49.770 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.003439663 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 16662, 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 17317 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 16662, 2024-06-03T01:24:49.771 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 488, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 16174, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 488, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 7172, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 325, 2024-06-03T01:24:49.772 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 8244, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-06-03T01:24:49.773 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:49.774 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-06-03T01:24:49.775 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-06-03T01:24:49.776 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-06-03T01:24:49.777 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-06-03T01:24:49.777 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 117, 2024-06-03T01:24:49.777 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-06-03T01:24:49.777 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-06-03T01:24:49.777 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-06-03T01:24:49.777 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-06-03T01:24:49.777 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 745, 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 313 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:49.778 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 10, 2024-06-03T01:24:49.779 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 5, 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 8224, 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 4112, 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:49.780 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 93754, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 22822517, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 93754, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 93754, 2024-06-03T01:24:49.781 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 22822517, 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:49.782 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 16662, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 35974111, 2024-06-03T01:24:49.783 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 16662, 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 35974111, 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:49.784 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 16662, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 16662, 2024-06-03T01:24:49.785 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 16662, 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 16662, 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.786 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 4112, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 448656, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 4112, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:49.787 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 488, 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 448656, 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:49.788 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x5578f1a10500": { 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 4930, 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 21339, 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 34646745, 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 21339, 2024-06-03T01:24:49.789 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 6684, 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 34641815, 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:49.790 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.791 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:49.791 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-06-03T01:24:50.492 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-06-03T01:24:50.492 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-06-03T01:24:50.492 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-06-03T01:24:50.493 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:50.493 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:50.493 DEBUG:tasks.check_counter:Getting stats from l 2024-06-03T01:24:50.493 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-03T01:24:50.730 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.c/config 2024-06-03T01:24:50.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:50 smithi094 ceph-mon[32119]: pgmap v453: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 501 KiB/s wr, 21 op/s 2024-06-03T01:24:50.851 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:50 smithi179 ceph-mon[37041]: pgmap v453: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 501 KiB/s wr, 21 op/s 2024-06-03T01:24:51.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:50 smithi107 ceph-mon[36622]: pgmap v453: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 501 KiB/s wr, 21 op/s 2024-06-03T01:24:52.448 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-06-03T01:24:52.455 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-06-03T01:24:52.456 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2873, 2024-06-03T01:24:52.456 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2875, 2024-06-03T01:24:52.456 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 22886184, 2024-06-03T01:24:52.456 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 913621, 2024-06-03T01:24:52.456 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 7, 2024-06-03T01:24:52.456 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-06-03T01:24:52.456 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.248052426, 2024-06-03T01:24:52.457 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.076739636, 2024-06-03T01:24:52.457 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.071825143, 2024-06-03T01:24:52.457 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.079718213, 2024-06-03T01:24:52.457 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:52.458 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2873, 2024-06-03T01:24:52.458 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.099755888, 2024-06-03T01:24:52.458 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000034721 2024-06-03T01:24:52.458 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.458 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:52.458 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.458 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 462, 2024-06-03T01:24:52.459 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 716, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 8510016, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 722790, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 7, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.058015906, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.019611993, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.018373399, 2024-06-03T01:24:52.460 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.014025308, 2024-06-03T01:24:52.461 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:52.461 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 713, 2024-06-03T01:24:52.461 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.042664213, 2024-06-03T01:24:52.461 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000059837 2024-06-03T01:24:52.461 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.461 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:52.461 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 576448 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-06-03T01:24:52.462 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 675, 2024-06-03T01:24:52.463 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 616, 2024-06-03T01:24:52.463 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 6221446, 2024-06-03T01:24:52.463 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 292672, 2024-06-03T01:24:52.463 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 5, 2024-06-03T01:24:52.463 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-06-03T01:24:52.463 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.087296614, 2024-06-03T01:24:52.463 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.025930021, 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.053638480, 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.015142072, 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 616, 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.025936260, 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000042104 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.464 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-06-03T01:24:52.465 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.465 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.465 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.465 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.465 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1962416, 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 211264 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-06-03T01:24:52.466 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 6110, 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.152352500, 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000024934 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-06-03T01:24:52.467 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-06-03T01:24:52.468 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-06-03T01:24:52.468 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.468 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.468 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.468 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.469 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.469 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-06-03T01:24:52.469 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-06-03T01:24:52.469 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-06-03T01:24:52.469 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-06-03T01:24:52.469 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.469 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.470 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.470 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.470 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-06-03T01:24:52.470 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-06-03T01:24:52.470 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-06-03T01:24:52.470 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-06-03T01:24:52.470 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-06-03T01:24:52.471 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-06-03T01:24:52.471 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-06-03T01:24:52.471 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 4920, 2024-06-03T01:24:52.471 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 363, 2024-06-03T01:24:52.471 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 4557, 2024-06-03T01:24:52.471 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-06-03T01:24:52.471 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 4402, 2024-06-03T01:24:52.472 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-06-03T01:24:52.472 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-06-03T01:24:52.472 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-06-03T01:24:52.472 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 7, 2024-06-03T01:24:52.472 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-06-03T01:24:52.472 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-06-03T01:24:52.473 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-06-03T01:24:52.473 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-06-03T01:24:52.473 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-06-03T01:24:52.473 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-06-03T01:24:52.473 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-06-03T01:24:52.473 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-06-03T01:24:52.473 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-06-03T01:24:52.474 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-06-03T01:24:52.475 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-06-03T01:24:52.476 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-06-03T01:24:52.476 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-06-03T01:24:52.476 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-06-03T01:24:52.476 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-06-03T01:24:52.476 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-06-03T01:24:52.476 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.477 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-06-03T01:24:52.478 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-06-03T01:24:52.479 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-06-03T01:24:52.479 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-06-03T01:24:52.479 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-06-03T01:24:52.479 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-06-03T01:24:52.479 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-06-03T01:24:52.479 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-06-03T01:24:52.480 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-06-03T01:24:52.480 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-06-03T01:24:52.480 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-06-03T01:24:52.480 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-06-03T01:24:52.480 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-06-03T01:24:52.480 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-06-03T01:24:52.480 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-06-03T01:24:52.481 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-06-03T01:24:52.482 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-06-03T01:24:52.482 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 21339, 2024-06-03T01:24:52.482 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-06-03T01:24:52.482 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-06-03T01:24:52.482 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-06-03T01:24:52.482 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-06-03T01:24:52.482 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 27, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 3, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 38841049, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 38840475, 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-06-03T01:24:52.483 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 21339 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 628, 2024-06-03T01:24:52.484 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 4854, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 4226, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 48, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 61, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 13, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 4920, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 5101, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 181, 2024-06-03T01:24:52.485 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-06-03T01:24:52.486 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-06-03T01:24:52.486 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-06-03T01:24:52.486 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 59824, 2024-06-03T01:24:52.486 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215420 2024-06-03T01:24:52.486 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.486 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-06-03T01:24:52.487 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-06-03T01:24:52.487 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-06-03T01:24:52.487 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-06-03T01:24:52.487 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-06-03T01:24:52.487 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-06-03T01:24:52.487 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-06-03T01:24:52.487 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.488 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.488 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.488 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.488 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-06-03T01:24:52.488 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.488 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.488 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.489 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.489 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-06-03T01:24:52.489 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.489 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.489 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.489 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.490 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-06-03T01:24:52.491 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.492 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.493 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-06-03T01:24:52.493 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.493 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.493 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.493 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.493 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-06-03T01:24:52.493 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.494 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.494 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.494 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.494 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-06-03T01:24:52.494 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.494 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.495 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.495 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.495 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.496 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.497 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.497 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-06-03T01:24:52.497 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.497 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.497 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.497 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.498 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.499 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.500 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.501 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.502 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.502 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.502 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-06-03T01:24:52.502 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.502 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.502 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-06-03T01:24:52.503 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.504 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.505 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.506 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.506 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-06-03T01:24:52.506 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.506 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.506 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.506 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.506 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-06-03T01:24:52.507 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.507 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.507 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.507 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.507 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-06-03T01:24:52.507 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-06-03T01:24:52.507 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-06-03T01:24:52.508 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.508 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.508 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.508 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.508 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.508 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-06-03T01:24:52.508 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-06-03T01:24:52.509 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 3, 2024-06-03T01:24:52.509 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-06-03T01:24:52.509 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-06-03T01:24:52.509 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-06-03T01:24:52.509 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-06-03T01:24:52.509 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-06-03T01:24:52.509 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 2701, 2024-06-03T01:24:52.510 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-06-03T01:24:52.510 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.510 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-06-03T01:24:52.511 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-06-03T01:24:52.511 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-06-03T01:24:52.511 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-06-03T01:24:52.511 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-06-03T01:24:52.511 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-06-03T01:24:52.511 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-03T01:24:52.512 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-03T01:24:52.513 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-06-03T01:24:52.514 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-06-03T01:24:52.514 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-06-03T01:24:52.514 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-06-03T01:24:52.514 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-06-03T01:24:52.514 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-03T01:24:52.514 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-06-03T01:24:52.515 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-06-03T01:24:52.515 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-06-03T01:24:52.515 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-06-03T01:24:52.515 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-06-03T01:24:52.515 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-06-03T01:24:52.515 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-06-03T01:24:52.515 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-06-03T01:24:52.516 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-06-03T01:24:52.516 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 233160, 2024-06-03T01:24:52.516 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 747, 2024-06-03T01:24:52.516 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-06-03T01:24:52.516 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-06-03T01:24:52.516 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-06-03T01:24:52.516 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9424, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 181, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-06-03T01:24:52.517 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-06-03T01:24:52.518 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-06-03T01:24:52.518 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-06-03T01:24:52.518 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 5102736, 2024-06-03T01:24:52.518 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 34481, 2024-06-03T01:24:52.518 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-06-03T01:24:52.518 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-06-03T01:24:52.519 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-06-03T01:24:52.519 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-06-03T01:24:52.519 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.519 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-06-03T01:24:52.519 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-06-03T01:24:52.519 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 3589, 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 21, 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 3, 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 3589, 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3589, 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.138950458, 2024-06-03T01:24:52.520 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000595973 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3589, 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3590 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 3589, 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 3589, 2024-06-03T01:24:52.521 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 2116, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1471, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-06-03T01:24:52.522 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-06-03T01:24:52.523 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 1, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-06-03T01:24:52.524 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-06-03T01:24:52.525 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-06-03T01:24:52.526 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-06-03T01:24:52.526 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-06-03T01:24:52.526 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-06-03T01:24:52.526 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-06-03T01:24:52.526 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-06-03T01:24:52.526 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 117, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-06-03T01:24:52.527 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 7, 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 8, 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 1, 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.528 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-06-03T01:24:52.529 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-06-03T01:24:52.529 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-06-03T01:24:52.529 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-06-03T01:24:52.529 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-06-03T01:24:52.529 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.529 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-06-03T01:24:52.529 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-06-03T01:24:52.530 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-06-03T01:24:52.531 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-06-03T01:24:52.531 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.531 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-03T01:24:52.531 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:52.531 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 4006, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 37232994, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 4006, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:52.532 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4006, 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 37232994, 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-06-03T01:24:52.533 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:52.534 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-06-03T01:24:52.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:52.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:52.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:52.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:52.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3589, 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 34652826, 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3589, 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 34652826, 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.535 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:52.536 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3589, 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 3589, 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3589, 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 3589, 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:52.537 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:52.538 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-06-03T01:24:52.539 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55729b14a6e0": { 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-06-03T01:24:52.540 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-06-03T01:24:52.541 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-06-03T01:24:52.541 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-06-03T01:24:52.541 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-06-03T01:24:52.541 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-06-03T01:24:52.541 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-06-03T01:24:52.541 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-06-03T01:24:52.541 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-06-03T01:24:52.542 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-06-03T01:24:52.542 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-06-03T01:24:52.542 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-06-03T01:24:52.542 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-06-03T01:24:52.542 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-06-03T01:24:52.542 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-06-03T01:24:52.543 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.543 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-06-03T01:24:52.543 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-06-03T01:24:52.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:52 smithi094 ceph-mon[32119]: pgmap v454: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 501 KiB/s wr, 21 op/s 2024-06-03T01:24:52.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:52 smithi179 ceph-mon[37041]: pgmap v454: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 501 KiB/s wr, 21 op/s 2024-06-03T01:24:53.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:52 smithi107 ceph-mon[36622]: pgmap v454: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 501 KiB/s wr, 21 op/s 2024-06-03T01:24:53.273 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-03T01:24:53.273 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-03T01:24:53.273 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-06-03T01:24:53.273 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-06-03T01:24:53.273 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-06-03T01:24:53.274 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-03T01:24:53.293 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-03T01:24:53.294 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T01:24:53.294 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:24:53.355 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-06-03T01:24:53.355 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-03T01:24:53.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-03T01:24:53.862 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:24:54.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:54 smithi094 ceph-mon[32119]: pgmap v455: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 507 KiB/s wr, 27 op/s 2024-06-03T01:24:54.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:54 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:54 smithi179 ceph-mon[37041]: pgmap v455: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 507 KiB/s wr, 27 op/s 2024-06-03T01:24:54.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:54 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:55.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:54 smithi107 ceph-mon[36622]: pgmap v455: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 507 KiB/s wr, 27 op/s 2024-06-03T01:24:55.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:54 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-03T01:24:55.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:55 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:55.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:55 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:55.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:55 smithi094 ceph-mon[32119]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:24:55.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:55 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:55.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:55 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:55.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:55 smithi179 ceph-mon[37041]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:24:56.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:55 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-03T01:24:56.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:55 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-03T01:24:56.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:55 smithi107 ceph-mon[36622]: from='mgr.15084 ' entity='mgr.x' 2024-06-03T01:24:56.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 -- 172.21.15.94:0/2788842479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 msgr2=0x7fea6410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:56.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 --2- 172.21.15.94:0/2788842479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 0x7fea6410f3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00b0d0 tx=0x7fea5c00b3e0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 -- 172.21.15.94:0/2788842479 shutdown_connections 2024-06-03T01:24:56.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 --2- 172.21.15.94:0/2788842479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 0x7fea6410f3c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 --2- 172.21.15.94:0/2788842479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea64102740 0x7fea64102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 --2- 172.21.15.94:0/2788842479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 0x7fea64107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 -- 172.21.15.94:0/2788842479 >> 172.21.15.94:0/2788842479 conn(0x7fea640795f0 msgr2=0x7fea64079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 -- 172.21.15.94:0/2788842479 shutdown_connections 2024-06-03T01:24:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.314+0000 7fea6ae00700 1 -- 172.21.15.94:0/2788842479 wait complete. 2024-06-03T01:24:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.315+0000 7fea6ae00700 1 Processor -- start 2024-06-03T01:24:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.316+0000 7fea6ae00700 1 -- start start 2024-06-03T01:24:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.316+0000 7fea6ae00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea64102740 0x7fea6419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.316+0000 7fea6ae00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 0x7fea6419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.316+0000 7fea6ae00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 0x7fea6419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.316+0000 7fea6ae00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112780 con 0x7fea64102740 2024-06-03T01:24:56.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.316+0000 7fea6ae00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112600 con 0x7fea64107840 2024-06-03T01:24:56.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.316+0000 7fea6ae00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112900 con 0x7fea641030f0 2024-06-03T01:24:56.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea64102740 0x7fea6419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:56.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 0x7fea6419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:56.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 0x7fea6419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53048/0 (socket says 172.21.15.94:53048) 2024-06-03T01:24:56.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 -- 172.21.15.94:0/3498625010 learned_addr learned my addr 172.21.15.94:0/3498625010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:24:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea68e00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 0x7fea6419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 -- 172.21.15.94:0/3498625010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 msgr2=0x7fea6419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 0x7fea6419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 -- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea64102740 msgr2=0x7fea6419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea64102740 0x7fea6419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 -- 172.21.15.94:0/3498625010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6419fd60 con 0x7fea641030f0 2024-06-03T01:24:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.317+0000 7fea63400700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 0x7fea6419b290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e9c0 tx=0x7fea5000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:56.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.318+0000 7fea60c00700 1 -- 172.21.15.94:0/3498625010 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5000cbd0 con 0x7fea641030f0 2024-06-03T01:24:56.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.318+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea641a0050 con 0x7fea641030f0 2024-06-03T01:24:56.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.318+0000 7fea60c00700 1 -- 172.21.15.94:0/3498625010 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5000ee60 con 0x7fea641030f0 2024-06-03T01:24:56.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.318+0000 7fea60c00700 1 -- 172.21.15.94:0/3498625010 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50005630 con 0x7fea641030f0 2024-06-03T01:24:56.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.318+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a78d0 con 0x7fea641030f0 2024-06-03T01:24:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.319+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea44005330 con 0x7fea641030f0 2024-06-03T01:24:56.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.322+0000 7fea60c00700 1 -- 172.21.15.94:0/3498625010 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fea50004830 con 0x7fea641030f0 2024-06-03T01:24:56.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.323+0000 7fea60c00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fea480a2d90 0x7fea480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:24:56.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.323+0000 7fea60c00700 1 -- 172.21.15.94:0/3498625010 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7fea500cfa20 con 0x7fea641030f0 2024-06-03T01:24:56.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.323+0000 7fea63e00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fea480a2d90 0x7fea480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:24:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.324+0000 7fea63e00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fea480a2d90 0x7fea480a5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea54009c60 tx=0x7fea54009260 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:24:56.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.328+0000 7fea60c00700 1 -- 172.21.15.94:0/3498625010 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea5000b2c0 con 0x7fea641030f0 2024-06-03T01:24:56.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.569+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea440058f0 con 0x7fea641030f0 2024-06-03T01:24:56.569 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:56 smithi094 ceph-mon[32119]: pgmap v456: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-06-03T01:24:56.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.569+0000 7fea60c00700 1 -- 172.21.15.94:0/3498625010 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v117) ==== 81+0+942 (secure 0 0 0) 0x7fea5009c220 con 0x7fea641030f0 2024-06-03T01:24:56.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/549573361 2024-06-04T01:10:20.680471+0000 2024-06-03T01:24:56.570 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2999518385 2024-06-04T01:10:20.680471+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/4120992823 2024-06-04T01:10:20.680471+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3717514300 2024-06-04T01:10:20.680471+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/127652834 2024-06-04T00:56:34.111978+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1904227986 2024-06-04T01:10:20.680471+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/4120992823 2024-06-04T01:10:20.680471+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/127652834 2024-06-04T00:56:34.111978+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2902022794 2024-06-04T00:56:34.111978+0000 2024-06-03T01:24:56.571 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3243526621 2024-06-04T00:55:59.358088+0000 2024-06-03T01:24:56.572 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2467202876 2024-06-04T00:55:59.358088+0000 2024-06-03T01:24:56.572 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2467202876 2024-06-04T00:55:59.358088+0000 2024-06-03T01:24:56.572 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3121579087 2024-06-04T00:55:59.358088+0000 2024-06-03T01:24:56.572 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1105337314 2024-06-04T00:55:59.358088+0000 2024-06-03T01:24:56.572 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3874005975 2024-06-04T00:56:34.111978+0000 2024-06-03T01:24:56.572 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/516090952 2024-06-04T00:56:34.111978+0000 2024-06-03T01:24:56.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fea480a2d90 msgr2=0x7fea480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:56.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fea480a2d90 0x7fea480a5250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea54009c60 tx=0x7fea54009260 comp rx=0 tx=0).stop 2024-06-03T01:24:56.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 msgr2=0x7fea6419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:24:56.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 0x7fea6419b290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e9c0 tx=0x7fea5000ecd0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 shutdown_connections 2024-06-03T01:24:56.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fea480a2d90 0x7fea480a5250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64107840 0x7fea6419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea64102740 0x7fea6419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 --2- 172.21.15.94:0/3498625010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea641030f0 0x7fea6419b290 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:24:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 >> 172.21.15.94:0/3498625010 conn(0x7fea640795f0 msgr2=0x7fea641006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:24:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 shutdown_connections 2024-06-03T01:24:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:24:56.573+0000 7fea6ae00700 1 -- 172.21.15.94:0/3498625010 wait complete. 2024-06-03T01:24:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:listed 16 entries 2024-06-03T01:24:56.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:56 smithi179 ceph-mon[37041]: pgmap v456: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-06-03T01:24:57.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:56 smithi107 ceph-mon[36622]: pgmap v456: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-06-03T01:24:57.488 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-03T01:24:57.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:24:57.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-03T01:24:57.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:57 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3498625010' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:24:57.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:24:57.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:24:57.707 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-03T01:24:57.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:24:57.786 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-06-03T01:24:57.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:57 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3498625010' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:24:57.858 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-03T01:24:57.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:24:57.860 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-03T01:24:57.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-03T01:24:57.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-06-03T01:24:58.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:57 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3498625010' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-03T01:24:58.044 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-03T01:24:58.054 INFO:tasks.cephadm:Teardown begin 2024-06-03T01:24:58.055 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-03T01:24:58.055 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:24:58.445 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:24:58.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:24:58 smithi094 ceph-mon[32119]: pgmap v457: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 20 op/s 2024-06-03T01:24:58.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:24:58 smithi179 ceph-mon[37041]: pgmap v457: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 20 op/s 2024-06-03T01:24:59.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:24:58 smithi107 ceph-mon[36622]: pgmap v457: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 60 KiB/s wr, 20 op/s 2024-06-03T01:25:00.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.788+0000 7f390fa00700 1 -- 172.21.15.94:0/1654593149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 msgr2=0x7f390806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:00.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.788+0000 7f390fa00700 1 --2- 172.21.15.94:0/1654593149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390806a5f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009990 tx=0x7f38f4009ca0 comp rx=0 tx=0).stop 2024-06-03T01:25:00.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.788+0000 7f390fa00700 1 -- 172.21.15.94:0/1654593149 shutdown_connections 2024-06-03T01:25:00.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.788+0000 7f390fa00700 1 --2- 172.21.15.94:0/1654593149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908108500 0x7f39081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:00.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.788+0000 7f390fa00700 1 --2- 172.21.15.94:0/1654593149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390806ab30 0x7f390810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:00.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.788+0000 7f390fa00700 1 --2- 172.21.15.94:0/1654593149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390806a5f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:00.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.788+0000 7f390fa00700 1 -- 172.21.15.94:0/1654593149 >> 172.21.15.94:0/1654593149 conn(0x7f3908078340 msgr2=0x7f3908078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:00.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.789+0000 7f390fa00700 1 -- 172.21.15.94:0/1654593149 shutdown_connections 2024-06-03T01:25:00.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.789+0000 7f390fa00700 1 -- 172.21.15.94:0/1654593149 wait complete. 2024-06-03T01:25:00.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.790+0000 7f390fa00700 1 Processor -- start 2024-06-03T01:25:00.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.790+0000 7f390fa00700 1 -- start start 2024-06-03T01:25:00.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390fa00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:00.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390fa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390806ab30 0x7f390819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:00.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390fa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908108500 0x7f390819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:00.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390fa00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39081127d0 con 0x7f390806ab30 2024-06-03T01:25:00.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390fa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3908112650 con 0x7f3908108500 2024-06-03T01:25:00.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390fa00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3908112950 con 0x7f390806a170 2024-06-03T01:25:00.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390d000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:00.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390d000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.94:53070/0 (socket says 172.21.15.94:53070) 2024-06-03T01:25:00.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390d000700 1 -- 172.21.15.94:0/4167469001 learned_addr learned my addr 172.21.15.94:0/4167469001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:00.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.791+0000 7f390da00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908108500 0x7f390819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f390d000700 1 -- 172.21.15.94:0/4167469001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908108500 msgr2=0x7f390819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f390d000700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908108500 0x7f390819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f390d000700 1 -- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390806ab30 msgr2=0x7f390819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f3907e00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390806ab30 0x7f390819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f390d000700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390806ab30 0x7f390819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:00.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f390d000700 1 -- 172.21.15.94:0/4167469001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390819fe70 con 0x7f390806a170 2024-06-03T01:25:00.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f3907e00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390806ab30 0x7f390819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-03T01:25:00.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.792+0000 7f390d000700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390819ae60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00d980 tx=0x7f38fc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:00.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.793+0000 7f3905600700 1 -- 172.21.15.94:0/4167469001 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc017070 con 0x7f390806a170 2024-06-03T01:25:00.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.793+0000 7f3905600700 1 -- 172.21.15.94:0/4167469001 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38fc005020 con 0x7f390806a170 2024-06-03T01:25:00.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.793+0000 7f3905600700 1 -- 172.21.15.94:0/4167469001 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc009bc0 con 0x7f390806a170 2024-06-03T01:25:00.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.793+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39081a0160 con 0x7f390806a170 2024-06-03T01:25:00.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.793+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a79e0 con 0x7f390806a170 2024-06-03T01:25:00.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.794+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3908069180 con 0x7f390806a170 2024-06-03T01:25:00.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.797+0000 7f3905600700 1 -- 172.21.15.94:0/4167469001 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f38fc0051d0 con 0x7f390806a170 2024-06-03T01:25:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.798+0000 7f3905600700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38f00a2e10 0x7f38f00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:00.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.798+0000 7f3905600700 1 -- 172.21.15.94:0/4167469001 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7f38fc0cfab0 con 0x7f390806a170 2024-06-03T01:25:00.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.799+0000 7f3907e00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38f00a2e10 0x7f38f00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:00.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.800+0000 7f3907e00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38f00a2e10 0x7f38f00a52d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009750 tx=0x7f38f4005fb0 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:00.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:00.801+0000 7f3905600700 1 -- 172.21.15.94:0/4167469001 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f38fc020020 con 0x7f390806a170 2024-06-03T01:25:00.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:00 smithi094 ceph-mon[32119]: pgmap v458: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-03T01:25:00.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:00 smithi179 ceph-mon[37041]: pgmap v458: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-03T01:25:01.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:00 smithi107 ceph-mon[36622]: pgmap v458: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-03T01:25:01.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.086+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f390819c1c0 con 0x7f390806a170 2024-06-03T01:25:01.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.087+0000 7f3905600700 1 -- 172.21.15.94:0/4167469001 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+11963 (secure 0 0 0) 0x7f38fc09c330 con 0x7f390806a170 2024-06-03T01:25:01.088 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:25:01.088 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:24:42.370665+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24490,24478,24485]},"id":1}]} 2024-06-03T01:25:01.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38f00a2e10 msgr2=0x7f38f00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:01.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38f00a2e10 0x7f38f00a52d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009750 tx=0x7f38f4005fb0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 msgr2=0x7f390819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:01.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390819ae60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00d980 tx=0x7f38fc00dc90 comp rx=0 tx=0).stop 2024-06-03T01:25:01.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 shutdown_connections 2024-06-03T01:25:01.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f38f00a2e10 0x7f38f00a52d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3908108500 0x7f390819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f390806a170 0x7f390819ae60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 --2- 172.21.15.94:0/4167469001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f390806ab30 0x7f390819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:01.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 >> 172.21.15.94:0/4167469001 conn(0x7f3908078340 msgr2=0x7f39080ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:01.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.090+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 shutdown_connections 2024-06-03T01:25:01.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:01.091+0000 7f390fa00700 1 -- 172.21.15.94:0/4167469001 wait complete. 2024-06-03T01:25:01.092 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 72 2024-06-03T01:25:01.737 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:01 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/4167469001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:25:01.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:01 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/4167469001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:25:02.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:01 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/4167469001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:25:02.236 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-03T01:25:02.690 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:25:02.834 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:02 smithi094 ceph-mon[32119]: pgmap v459: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-03T01:25:02.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:02 smithi179 ceph-mon[37041]: pgmap v459: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-03T01:25:03.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:02 smithi107 ceph-mon[36622]: pgmap v459: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-03T01:25:04.819 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:04 smithi094 ceph-mon[32119]: pgmap v460: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 35 op/s 2024-06-03T01:25:04.835 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:04 smithi107 ceph-mon[36622]: pgmap v460: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 35 op/s 2024-06-03T01:25:04.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:04 smithi179 ceph-mon[37041]: pgmap v460: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 35 op/s 2024-06-03T01:25:05.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 -- 172.21.15.94:0/2193286501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 msgr2=0x7f8a00102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:05.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 --2- 172.21.15.94:0/2193286501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00102b20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f89f00099c0 tx=0x7f89f0009cd0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 -- 172.21.15.94:0/2193286501 shutdown_connections 2024-06-03T01:25:05.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 --2- 172.21.15.94:0/2193286501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a00107840 0x7f8a0010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 --2- 172.21.15.94:0/2193286501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00102b20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 --2- 172.21.15.94:0/2193286501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a001030f0 0x7f8a00107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 -- 172.21.15.94:0/2193286501 >> 172.21.15.94:0/2193286501 conn(0x7f8a000795f0 msgr2=0x7f8a00079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:05.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 -- 172.21.15.94:0/2193286501 shutdown_connections 2024-06-03T01:25:05.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.117+0000 7f8a07200700 1 -- 172.21.15.94:0/2193286501 wait complete. 2024-06-03T01:25:05.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.118+0000 7f8a07200700 1 Processor -- start 2024-06-03T01:25:05.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.118+0000 7f8a07200700 1 -- start start 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.118+0000 7f8a07200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.118+0000 7f8a07200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a001030f0 0x7f8a00074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f8a07200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00107840 0x7f8a00075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f8a07200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00112760 con 0x7f8a00102740 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f8a07200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001125e0 con 0x7f8a00107840 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f8a07200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001128e0 con 0x7f8a001030f0 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:05.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48576/0 (socket says 172.21.15.94:48576) 2024-06-03T01:25:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 -- 172.21.15.94:0/2023925008 learned_addr learned my addr 172.21.15.94:0/2023925008 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89f7400700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a001030f0 0x7f8a00074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f8a05200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00107840 0x7f8a00075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 -- 172.21.15.94:0/2023925008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a001030f0 msgr2=0x7f8a00074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a001030f0 0x7f8a00074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 -- 172.21.15.94:0/2023925008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00107840 msgr2=0x7f8a00075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:05.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00107840 0x7f8a00075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f89ffe00700 1 -- 172.21.15.94:0/2023925008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a001a70f0 con 0x7f8a00102740 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.119+0000 7f8a05200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00107840 0x7f8a00075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.120+0000 7f89ffe00700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00074790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006010 tx=0x7f89f000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.120+0000 7f89fd600700 1 -- 172.21.15.94:0/2023925008 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0003680 con 0x7f8a00102740 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.120+0000 7f89fd600700 1 -- 172.21.15.94:0/2023925008 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f00055b0 con 0x7f8a00102740 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.120+0000 7f89fd600700 1 -- 172.21.15.94:0/2023925008 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00173f0 con 0x7f8a00102740 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.120+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a001a7380 con 0x7f8a00102740 2024-06-03T01:25:05.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.120+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a001a7780 con 0x7f8a00102740 2024-06-03T01:25:05.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.121+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0010aa80 con 0x7f8a00102740 2024-06-03T01:25:05.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.123+0000 7f89fd600700 1 -- 172.21.15.94:0/2023925008 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f89f000fca0 con 0x7f8a00102740 2024-06-03T01:25:05.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.125+0000 7f89fd600700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f89e40a2fb0 0x7f89e40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:05.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.125+0000 7f89fd600700 1 -- 172.21.15.94:0/2023925008 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7f89f00d06e0 con 0x7f8a00102740 2024-06-03T01:25:05.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.125+0000 7f89f7400700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f89e40a2fb0 0x7f89e40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:05.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.126+0000 7f89f7400700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f89e40a2fb0 0x7f89e40a5470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8a00100f60 tx=0x7f89e8009260 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:05.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.129+0000 7f89fd600700 1 -- 172.21.15.94:0/2023925008 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f89f0013070 con 0x7f8a00102740 2024-06-03T01:25:05.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.401+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a00069180 con 0x7f8a00102740 2024-06-03T01:25:05.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.404+0000 7f89fd600700 1 -- 172.21.15.94:0/2023925008 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+11963 (secure 0 0 0) 0x7f89f009cf70 con 0x7f8a00102740 2024-06-03T01:25:05.406 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:25:05.406 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3523136446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3523136446},{"type":"v1","addr":"172.21.15.94:6835","nonce":3523136446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3219738188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":3219738188},{"type":"v1","addr":"172.21.15.94:6837","nonce":3219738188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-03T01:04:57.629349+0000","modified":"2024-06-03T01:24:42.370665+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24505,"mds_2":24490,"mds_3":24485,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14388":{"gid":14388,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/848723917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":848723917},{"type":"v1","addr":"172.21.15.94:6839","nonce":848723917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14400":{"gid":14400,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6841/3437199777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":3437199777},{"type":"v1","addr":"172.21.15.94:6841","nonce":3437199777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6833/1330319563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1330319563},{"type":"v1","addr":"172.21.15.107:6833","nonce":1330319563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/2660538909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2660538909},{"type":"v1","addr":"172.21.15.179:6833","nonce":2660538909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6835/2528321857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2528321857},{"type":"v1","addr":"172.21.15.107:6835","nonce":2528321857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":4,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.179:6835/1407039746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":1407039746},{"type":"v1","addr":"172.21.15.179:6835","nonce":1407039746}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.107:6837/2537813051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2537813051},{"type":"v1","addr":"172.21.15.107:6837","nonce":2537813051}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":31,"addr":"172.21.15.179:6837/2233357781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2233357781},{"type":"v1","addr":"172.21.15.179:6837","nonce":2233357781}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":30,"addr":"172.21.15.179:6839/544081992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":544081992},{"type":"v1","addr":"172.21.15.179:6839","nonce":544081992}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.107:6839/1423733861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1423733861},{"type":"v1","addr":"172.21.15.107:6839","nonce":1423733861}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24490,24478,24485]},"id":1}]} 2024-06-03T01:25:05.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.407+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f89e40a2fb0 msgr2=0x7f89e40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:05.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.407+0000 7f8a07200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f89e40a2fb0 0x7f89e40a5470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8a00100f60 tx=0x7f89e8009260 comp rx=0 tx=0).stop 2024-06-03T01:25:05.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.407+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 msgr2=0x7f8a00074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:05.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.407+0000 7f8a07200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00074790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f89f0006010 tx=0x7f89f000f690 comp rx=0 tx=0).stop 2024-06-03T01:25:05.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.407+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 shutdown_connections 2024-06-03T01:25:05.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.408+0000 7f8a07200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f89e40a2fb0 0x7f89e40a5470 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.408+0000 7f8a07200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00107840 0x7f8a00075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.408+0000 7f8a07200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a00102740 0x7f8a00074790 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.408+0000 7f8a07200700 1 --2- 172.21.15.94:0/2023925008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a001030f0 0x7f8a00074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:05.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.408+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 >> 172.21.15.94:0/2023925008 conn(0x7f8a000795f0 msgr2=0x7f8a00101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:05.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.408+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 shutdown_connections 2024-06-03T01:25:05.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:05.408+0000 7f8a07200700 1 -- 172.21.15.94:0/2023925008 wait complete. 2024-06-03T01:25:05.410 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 72 2024-06-03T01:25:05.830 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:05 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2023925008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:25:05.851 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:05 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2023925008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:25:05.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:05 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2023925008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-03T01:25:06.281 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-03T01:25:06.725 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:25:06.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:06 smithi094 ceph-mon[32119]: pgmap v461: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 29 op/s 2024-06-03T01:25:06.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:06 smithi179 ceph-mon[37041]: pgmap v461: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 29 op/s 2024-06-03T01:25:07.031 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:06 smithi107 ceph-mon[36622]: pgmap v461: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 29 op/s 2024-06-03T01:25:08.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:08 smithi094 ceph-mon[32119]: pgmap v462: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-06-03T01:25:08.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:08 smithi179 ceph-mon[37041]: pgmap v462: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-06-03T01:25:09.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:08 smithi107 ceph-mon[36622]: pgmap v462: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.184+0000 7fa6b1c00700 1 -- 172.21.15.94:0/3623611833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06a170 msgr2=0x7fa6ac06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.184+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/3623611833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06a170 0x7fa6ac06a5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa694009990 tx=0x7fa694009ca0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.184+0000 7fa6b1c00700 1 -- 172.21.15.94:0/3623611833 shutdown_connections 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.184+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/3623611833 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06ab30 0x7fa6ac10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.184+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/3623611833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac108500 0x7fa6ac1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.184+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/3623611833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06a170 0x7fa6ac06a5f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.184+0000 7fa6b1c00700 1 -- 172.21.15.94:0/3623611833 >> 172.21.15.94:0/3623611833 conn(0x7fa6ac078340 msgr2=0x7fa6ac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.185+0000 7fa6b1c00700 1 -- 172.21.15.94:0/3623611833 shutdown_connections 2024-06-03T01:25:09.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.185+0000 7fa6b1c00700 1 -- 172.21.15.94:0/3623611833 wait complete. 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.186+0000 7fa6b1c00700 1 Processor -- start 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.186+0000 7fa6b1c00700 1 -- start start 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6b1c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06a170 0x7fa6ac19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6b1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06ab30 0x7fa6ac19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:09.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6b1c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac108500 0x7fa6ac198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:09.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6b1c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac1127d0 con 0x7fa6ac06a170 2024-06-03T01:25:09.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6b1c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac112650 con 0x7fa6ac06ab30 2024-06-03T01:25:09.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6b1c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac112950 con 0x7fa6ac108500 2024-06-03T01:25:09.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6aaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06a170 0x7fa6ac19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:09.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6aaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06a170 0x7fa6ac19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48590/0 (socket says 172.21.15.94:48590) 2024-06-03T01:25:09.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6aaa00700 1 -- 172.21.15.94:0/1060888665 learned_addr learned my addr 172.21.15.94:0/1060888665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:09.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6a2000700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06ab30 0x7fa6ac19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.187+0000 7fa6ab400700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac108500 0x7fa6ac198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6aaa00700 1 -- 172.21.15.94:0/1060888665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac108500 msgr2=0x7fa6ac198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6aaa00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac108500 0x7fa6ac198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6aaa00700 1 -- 172.21.15.94:0/1060888665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06ab30 msgr2=0x7fa6ac19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6aaa00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06ab30 0x7fa6ac19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6aaa00700 1 -- 172.21.15.94:0/1060888665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac1991f0 con 0x7fa6ac06a170 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6a2000700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06ab30 0x7fa6ac19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:25:09.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6aaa00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06a170 0x7fa6ac19e8d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9c0 tx=0x7fa69c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6a3e00700 1 -- 172.21.15.94:0/1060888665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00cbd0 con 0x7fa6ac06a170 2024-06-03T01:25:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac1994e0 con 0x7fa6ac06a170 2024-06-03T01:25:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6a3e00700 1 -- 172.21.15.94:0/1060888665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69c00ee60 con 0x7fa6ac06a170 2024-06-03T01:25:09.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6a3e00700 1 -- 172.21.15.94:0/1060888665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c005630 con 0x7fa6ac06a170 2024-06-03T01:25:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.188+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1a57a0 con 0x7fa6ac06a170 2024-06-03T01:25:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.190+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac069180 con 0x7fa6ac06a170 2024-06-03T01:25:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.192+0000 7fa6a3e00700 1 -- 172.21.15.94:0/1060888665 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa69c004750 con 0x7fa6ac06a170 2024-06-03T01:25:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.193+0000 7fa6a3e00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa6900a2d90 0x7fa6900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.193+0000 7fa6a3e00700 1 -- 172.21.15.94:0/1060888665 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9475+0+0 (secure 0 0 0) 0x7fa69c0cfc80 con 0x7fa6ac06a170 2024-06-03T01:25:09.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.193+0000 7fa6a2000700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa6900a2d90 0x7fa6900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:09.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.194+0000 7fa6a2000700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa6900a2d90 0x7fa6900a5250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac078940 tx=0x7fa69400b260 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:09.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.196+0000 7fa6a3e00700 1 -- 172.21.15.94:0/1060888665 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa69c053850 con 0x7fa6ac06a170 2024-06-03T01:25:09.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.419+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6ac199fc0 con 0x7fa6ac06a170 2024-06-03T01:25:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.427+0000 7fa6a3e00700 1 -- 172.21.15.94:0/1060888665 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+25287 (secure 0 0 0) 0x7fa69c09c480 con 0x7fa6ac06a170 2024-06-03T01:25:09.428 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-06-03T01:25:09.429 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":117,"fsid":"d037ccd0-2143-11ef-bc9b-c7b262605968","created":"2024-06-03T00:55:25.857100+0000","modified":"2024-06-03T01:24:08.767371+0000","last_up_change":"2024-06-03T01:03:38.619066+0000","last_in_change":"2024-06-03T01:03:17.661080+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-03T00:59:20.779028+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-03T01:04:48.698832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-03T01:04:53.283627+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-03T01:05:06.876021+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"63946c12-7fdf-401c-9cbc-18db7ef2f714","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6803","nonce":837949984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6805","nonce":837949984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6809","nonce":837949984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":837949984},{"type":"v1","addr":"172.21.15.94:6807","nonce":837949984}]},"public_addr":"172.21.15.94:6803/837949984","cluster_addr":"172.21.15.94:6805/837949984","heartbeat_back_addr":"172.21.15.94:6809/837949984","heartbeat_front_addr":"172.21.15.94:6807/837949984","state":["exists","up"]},{"osd":1,"uuid":"6b19aeb7-1a47-4ed0-abdc-3d1e13704e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6801","nonce":3097634475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6803","nonce":3097634475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6807","nonce":3097634475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3097634475},{"type":"v1","addr":"172.21.15.107:6805","nonce":3097634475}]},"public_addr":"172.21.15.107:6801/3097634475","cluster_addr":"172.21.15.107:6803/3097634475","heartbeat_back_addr":"172.21.15.107:6807/3097634475","heartbeat_front_addr":"172.21.15.107:6805/3097634475","state":["exists","up"]},{"osd":2,"uuid":"a1f9c436-0127-4d60-9cfa-4a33e22409f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6801","nonce":3335947212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6803","nonce":3335947212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6807","nonce":3335947212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":3335947212},{"type":"v1","addr":"172.21.15.179:6805","nonce":3335947212}]},"public_addr":"172.21.15.179:6801/3335947212","cluster_addr":"172.21.15.179:6803/3335947212","heartbeat_back_addr":"172.21.15.179:6807/3335947212","heartbeat_front_addr":"172.21.15.179:6805/3335947212","state":["exists","up"]},{"osd":3,"uuid":"43ade6dd-20a3-419d-8dba-e718f61a022e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6811","nonce":667050825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6813","nonce":667050825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6817","nonce":667050825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":667050825},{"type":"v1","addr":"172.21.15.94:6815","nonce":667050825}]},"public_addr":"172.21.15.94:6811/667050825","cluster_addr":"172.21.15.94:6813/667050825","heartbeat_back_addr":"172.21.15.94:6817/667050825","heartbeat_front_addr":"172.21.15.94:6815/667050825","state":["exists","up"]},{"osd":4,"uuid":"7ed0339d-4210-4398-9092-db7bcba4048b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6809","nonce":1882504860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6811","nonce":1882504860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6815","nonce":1882504860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1882504860},{"type":"v1","addr":"172.21.15.107:6813","nonce":1882504860}]},"public_addr":"172.21.15.107:6809/1882504860","cluster_addr":"172.21.15.107:6811/1882504860","heartbeat_back_addr":"172.21.15.107:6815/1882504860","heartbeat_front_addr":"172.21.15.107:6813/1882504860","state":["exists","up"]},{"osd":5,"uuid":"b46c6631-7947-49fd-afa2-eb39f3145766","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6809","nonce":1536083025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6811","nonce":1536083025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6815","nonce":1536083025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1536083025},{"type":"v1","addr":"172.21.15.179:6813","nonce":1536083025}]},"public_addr":"172.21.15.179:6809/1536083025","cluster_addr":"172.21.15.179:6811/1536083025","heartbeat_back_addr":"172.21.15.179:6815/1536083025","heartbeat_front_addr":"172.21.15.179:6813/1536083025","state":["exists","up"]},{"osd":6,"uuid":"172b5d67-f3cf-48eb-9060-0b421fb38e63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6819","nonce":3118996469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6821","nonce":3118996469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6825","nonce":3118996469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3118996469},{"type":"v1","addr":"172.21.15.94:6823","nonce":3118996469}]},"public_addr":"172.21.15.94:6819/3118996469","cluster_addr":"172.21.15.94:6821/3118996469","heartbeat_back_addr":"172.21.15.94:6825/3118996469","heartbeat_front_addr":"172.21.15.94:6823/3118996469","state":["exists","up"]},{"osd":7,"uuid":"05b89eaa-5143-4517-949e-e25d75b4c5b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6817","nonce":2599842283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6819","nonce":2599842283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6823","nonce":2599842283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2599842283},{"type":"v1","addr":"172.21.15.107:6821","nonce":2599842283}]},"public_addr":"172.21.15.107:6817/2599842283","cluster_addr":"172.21.15.107:6819/2599842283","heartbeat_back_addr":"172.21.15.107:6823/2599842283","heartbeat_front_addr":"172.21.15.107:6821/2599842283","state":["exists","up"]},{"osd":8,"uuid":"07303dac-9a89-4c1a-8555-07075f29ed66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6817","nonce":3410192696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6819","nonce":3410192696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6823","nonce":3410192696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3410192696},{"type":"v1","addr":"172.21.15.179:6821","nonce":3410192696}]},"public_addr":"172.21.15.179:6817/3410192696","cluster_addr":"172.21.15.179:6819/3410192696","heartbeat_back_addr":"172.21.15.179:6823/3410192696","heartbeat_front_addr":"172.21.15.179:6821/3410192696","state":["exists","up"]},{"osd":9,"uuid":"126f498f-8208-42d7-bc3e-9dcc5f200906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6827","nonce":165849117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6829","nonce":165849117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6833","nonce":165849117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":165849117},{"type":"v1","addr":"172.21.15.94:6831","nonce":165849117}]},"public_addr":"172.21.15.94:6827/165849117","cluster_addr":"172.21.15.94:6829/165849117","heartbeat_back_addr":"172.21.15.94:6833/165849117","heartbeat_front_addr":"172.21.15.94:6831/165849117","state":["exists","up"]},{"osd":10,"uuid":"070f388a-c6f7-4438-86bc-57d8f08e4175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6825","nonce":1849339133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6827","nonce":1849339133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6831","nonce":1849339133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1849339133},{"type":"v1","addr":"172.21.15.107:6829","nonce":1849339133}]},"public_addr":"172.21.15.107:6825/1849339133","cluster_addr":"172.21.15.107:6827/1849339133","heartbeat_back_addr":"172.21.15.107:6831/1849339133","heartbeat_front_addr":"172.21.15.107:6829/1849339133","state":["exists","up"]},{"osd":11,"uuid":"d5ee1e38-10d2-4513-9d5b-29816f8d6eb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6825","nonce":4289321509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6827","nonce":4289321509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6831","nonce":4289321509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4289321509},{"type":"v1","addr":"172.21.15.179:6829","nonce":4289321509}]},"public_addr":"172.21.15.179:6825/4289321509","cluster_addr":"172.21.15.179:6827/4289321509","heartbeat_back_addr":"172.21.15.179:6831/4289321509","heartbeat_front_addr":"172.21.15.179:6829/4289321509","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:28.529562+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:58:53.468835+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:17.437790+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T00:59:50.498134+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:16.671501+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:00:41.791618+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:16.278167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:01:42.922271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:08.300829+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:02:42.940334+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:09.999466+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-03T01:03:35.889060+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/549573361":"2024-06-04T01:10:20.680471+0000","172.21.15.94:0/2999518385":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6800/4120992823":"2024-06-04T01:10:20.680471+0000","172.21.15.94:0/3717514300":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6801/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/1904227986":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6801/4120992823":"2024-06-04T01:10:20.680471+0000","172.21.15.94:6800/127652834":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/2902022794":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/3243526621":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6800/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:6801/2467202876":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3121579087":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/1105337314":"2024-06-04T00:55:59.358088+0000","172.21.15.94:0/3874005975":"2024-06-04T00:56:34.111978+0000","172.21.15.94:0/516090952":"2024-06-04T00:56:34.111978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]},{"pool":4,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-03T01:25:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.429+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa6900a2d90 msgr2=0x7fa6900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.429+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa6900a2d90 0x7fa6900a5250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac078940 tx=0x7fa69400b260 comp rx=0 tx=0).stop 2024-06-03T01:25:09.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.429+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06a170 msgr2=0x7fa6ac19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:09.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.429+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06a170 0x7fa6ac19e8d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00e9c0 tx=0x7fa69c00ecd0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 shutdown_connections 2024-06-03T01:25:09.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ac108500 0x7fa6ac198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6ac06a170 0x7fa6ac19e8d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa6900a2d90 0x7fa6900a5250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 --2- 172.21.15.94:0/1060888665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6ac06ab30 0x7fa6ac19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:09.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 >> 172.21.15.94:0/1060888665 conn(0x7fa6ac078340 msgr2=0x7fa6ac100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:09.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 shutdown_connections 2024-06-03T01:25:09.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:09.430+0000 7fa6b1c00700 1 -- 172.21.15.94:0/1060888665 wait complete. 2024-06-03T01:25:09.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:09 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1060888665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:25:10.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:09 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1060888665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:25:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:09 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1060888665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-03T01:25:10.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-03T01:25:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:25:10.913 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:10 smithi094 ceph-mon[32119]: pgmap v463: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-06-03T01:25:10.913 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:10 smithi094 ceph-mon[32119]: osdmap e118: 12 total, 12 up, 12 in 2024-06-03T01:25:11.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:10 smithi107 ceph-mon[36622]: pgmap v463: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-06-03T01:25:11.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:10 smithi107 ceph-mon[36622]: osdmap e118: 12 total, 12 up, 12 in 2024-06-03T01:25:11.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:10 smithi179 ceph-mon[37041]: pgmap v463: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 28 op/s 2024-06-03T01:25:11.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:10 smithi179 ceph-mon[37041]: osdmap e118: 12 total, 12 up, 12 in 2024-06-03T01:25:12.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:12 smithi094 ceph-mon[32119]: pgmap v465: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-06-03T01:25:13.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:12 smithi107 ceph-mon[36622]: pgmap v465: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-06-03T01:25:13.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:12 smithi179 ceph-mon[37041]: pgmap v465: 129 pgs: 129 active+clean; 359 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-06-03T01:25:13.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 -- 172.21.15.94:0/1453534543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 msgr2=0x7f5c9410e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:13.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 --2- 172.21.15.94:0/1453534543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9410e950 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5c800098f0 tx=0x7f5c80009c00 comp rx=0 tx=0).stop 2024-06-03T01:25:13.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 -- 172.21.15.94:0/1453534543 shutdown_connections 2024-06-03T01:25:13.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 --2- 172.21.15.94:0/1453534543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c9410ee90 0x7f5c94111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 --2- 172.21.15.94:0/1453534543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c940746c0 0x7f5c94074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 --2- 172.21.15.94:0/1453534543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9410e950 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 -- 172.21.15.94:0/1453534543 >> 172.21.15.94:0/1453534543 conn(0x7f5c94079530 msgr2=0x7f5c94079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:13.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 -- 172.21.15.94:0/1453534543 shutdown_connections 2024-06-03T01:25:13.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.142+0000 7f5c99200700 1 -- 172.21.15.94:0/1453534543 wait complete. 2024-06-03T01:25:13.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.143+0000 7f5c99200700 1 Processor -- start 2024-06-03T01:25:13.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.143+0000 7f5c99200700 1 -- start start 2024-06-03T01:25:13.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c99200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c940746c0 0x7f5c9419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:13.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c99200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c99200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c9410ee90 0x7f5c9419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c99200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94114980 con 0x7f5c94074fe0 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c99200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94114800 con 0x7f5c9410ee90 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c99200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94114b00 con 0x7f5c940746c0 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c91400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c91400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48608/0 (socket says 172.21.15.94:48608) 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c91400700 1 -- 172.21.15.94:0/2472813351 learned_addr learned my addr 172.21.15.94:0/2472813351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:13.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c92800700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c9410ee90 0x7f5c9419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.144+0000 7f5c91e00700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c940746c0 0x7f5c9419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c91400700 1 -- 172.21.15.94:0/2472813351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c940746c0 msgr2=0x7f5c9419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c91400700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c940746c0 0x7f5c9419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c91400700 1 -- 172.21.15.94:0/2472813351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c9410ee90 msgr2=0x7f5c9419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c91400700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c9410ee90 0x7f5c9419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c91400700 1 -- 172.21.15.94:0/2472813351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9419fe90 con 0x7f5c94074fe0 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c92800700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c9410ee90 0x7f5c9419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-03T01:25:13.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c91400700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9419b410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009230 tx=0x7f5c8000fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:13.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c8aa00700 1 -- 172.21.15.94:0/2472813351 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80005140 con 0x7f5c94074fe0 2024-06-03T01:25:13.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c8aa00700 1 -- 172.21.15.94:0/2472813351 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8000b560 con 0x7f5c94074fe0 2024-06-03T01:25:13.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.145+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c941a0120 con 0x7f5c94074fe0 2024-06-03T01:25:13.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.146+0000 7f5c8aa00700 1 -- 172.21.15.94:0/2472813351 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80003a50 con 0x7f5c94074fe0 2024-06-03T01:25:13.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.146+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c941a7a00 con 0x7f5c94074fe0 2024-06-03T01:25:13.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.147+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c94069180 con 0x7f5c94074fe0 2024-06-03T01:25:13.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.149+0000 7f5c8aa00700 1 -- 172.21.15.94:0/2472813351 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5c80029060 con 0x7f5c94074fe0 2024-06-03T01:25:13.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.150+0000 7f5c8aa00700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5c780a2e10 0x7f5c780a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:13.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.151+0000 7f5c8aa00700 1 -- 172.21.15.94:0/2472813351 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9563+0+0 (secure 0 0 0) 0x7f5c800d0060 con 0x7f5c94074fe0 2024-06-03T01:25:13.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.151+0000 7f5c91e00700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5c780a2e10 0x7f5c780a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:13.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.151+0000 7f5c91e00700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5c780a2e10 0x7f5c780a52d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005fd0 tx=0x7f5c84009310 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:13.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.155+0000 7f5c8aa00700 1 -- 172.21.15.94:0/2472813351 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5c80013030 con 0x7f5c94074fe0 2024-06-03T01:25:13.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.426+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5c9402dca0 con 0x7f5c94074fe0 2024-06-03T01:25:13.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.429+0000 7f5c8aa00700 1 -- 172.21.15.94:0/2472813351 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 161+0+0 (secure 0 0 0) 0x7f5c8009c8a0 con 0x7f5c94074fe0 2024-06-03T01:25:13.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.433+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5c780a2e10 msgr2=0x7f5c780a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.433+0000 7f5c99200700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5c780a2e10 0x7f5c780a52d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c84005fd0 tx=0x7f5c84009310 comp rx=0 tx=0).stop 2024-06-03T01:25:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.433+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 msgr2=0x7f5c9419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.433+0000 7f5c99200700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9419b410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009230 tx=0x7f5c8000fa70 comp rx=0 tx=0).stop 2024-06-03T01:25:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.434+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 shutdown_connections 2024-06-03T01:25:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.434+0000 7f5c99200700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f5c780a2e10 0x7f5c780a52d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.434+0000 7f5c99200700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c9410ee90 0x7f5c9419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.434+0000 7f5c99200700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c940746c0 0x7f5c9419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.434+0000 7f5c99200700 1 --2- 172.21.15.94:0/2472813351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c94074fe0 0x7f5c9419b410 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009230 tx=0x7f5c8000fa70 comp rx=0 tx=0).stop 2024-06-03T01:25:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.434+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 >> 172.21.15.94:0/2472813351 conn(0x7f5c94079530 msgr2=0x7f5c940ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.435+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 shutdown_connections 2024-06-03T01:25:13.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:13.435+0000 7f5c99200700 1 -- 172.21.15.94:0/2472813351 wait complete. 2024-06-03T01:25:13.440 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-03T01:25:13.783 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.783 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.784 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mds[67014]: -- 172.21.15.107:0/2707703965 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f40e461a40 con 0x55f40e434c00 2024-06-03T01:25:13.784 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: starting mds.b at 2024-06-03T01:25:13.784 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h[66699]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.784 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h[66699]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.784 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mds[66703]: -- 172.21.15.107:0/2080593910 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cdb2dc7a40 con 0x55cdb2d9b000 2024-06-03T01:25:13.784 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h[66699]: starting mds.h at 2024-06-03T01:25:13.785 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k[66083]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.785 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k[66083]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.785 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mds[66088]: -- 172.21.15.107:0/2126502521 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589ee3bfa40 con 0x5589ee392800 2024-06-03T01:25:13.785 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k[66083]: starting mds.k at 2024-06-03T01:25:13.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mon[36622]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:25:13.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2472813351' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T01:25:13.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mon[36622]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T01:25:13.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mon[36622]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:25:13.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2472813351' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T01:25:13.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mon[36622]: osdmap e119: 12 total, 12 up, 12 in 2024-06-03T01:25:13.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mon[36622]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T01:25:13.787 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e[66397]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.787 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e[66397]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.787 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-mds[66401]: -- 172.21.15.107:0/2255791125 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3515bba40 con 0x55c35158f000 2024-06-03T01:25:13.787 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:25:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e[66397]: starting mds.e at 2024-06-03T01:25:13.841 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a[74154]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.842 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a[74154]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.842 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mds[74158]: -- 172.21.15.94:0/1178052684 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b804df3a40 con 0x55b804dc6800 2024-06-03T01:25:13.842 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a[74154]: starting mds.a at 2024-06-03T01:25:13.842 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g[73749]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.842 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g[73749]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.843 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mds[73754]: -- 172.21.15.94:0/1489228581 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640ea2d5a40 con 0x5640ea2a8c00 2024-06-03T01:25:13.843 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g[73749]: starting mds.g at 2024-06-03T01:25:13.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a[32115]: 2024-06-03T01:25:13.427+0000 7fc8d0e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:25:13.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mon[32119]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:25:13.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2472813351' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T01:25:13.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mon[32119]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T01:25:13.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mon[32119]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:25:13.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2472813351' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T01:25:13.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mon[32119]: osdmap e119: 12 total, 12 up, 12 in 2024-06-03T01:25:13.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:13 smithi094 ceph-mon[32119]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T01:25:13.855 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i[67952]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.855 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i[67952]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.855 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mds[67956]: -- 172.21.15.179:0/2923787913 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582aa831a40 con 0x5582aa804800 2024-06-03T01:25:13.856 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i[67952]: starting mds.i at 2024-06-03T01:25:13.856 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f[67330]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.856 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f[67330]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.856 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mds[67334]: -- 172.21.15.179:0/972497033 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b54f8ada40 con 0x55b54f881000 2024-06-03T01:25:13.856 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f[67330]: starting mds.f at 2024-06-03T01:25:13.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mon[37041]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-03T01:25:13.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2472813351' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-03T01:25:13.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mon[37041]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-03T01:25:13.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mon[37041]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-03T01:25:13.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2472813351' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-03T01:25:13.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mon[37041]: osdmap e119: 12 total, 12 up, 12 in 2024-06-03T01:25:13.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mon[37041]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-03T01:25:13.858 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c[67635]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.858 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c[67635]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.858 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mds[67639]: -- 172.21.15.179:0/393271266 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618acb43a40 con 0x5618acb17000 2024-06-03T01:25:13.858 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c[67635]: starting mds.c at 2024-06-03T01:25:13.859 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l[67028]: ignoring --setuser ceph since I am not root 2024-06-03T01:25:13.859 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l[67028]: ignoring --setgroup ceph since I am not root 2024-06-03T01:25:13.859 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-mds[67032]: -- 172.21.15.179:0/179454629 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56284cd89a40 con 0x56284cd5c800 2024-06-03T01:25:13.859 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:25:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l[67028]: starting mds.l at 2024-06-03T01:25:14.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-03T01:25:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:25:14.913 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: pgmap v466: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-03T01:25:14.914 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: osdmap e120: 12 total, 12 up, 12 in 2024-06-03T01:25:14.914 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6832/2719959718,v1:172.21.15.107:6833/2719959718] up:boot 2024-06-03T01:25:14.914 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6832/2644203537,v1:172.21.15.179:6836/2644203537] up:boot 2024-06-03T01:25:14.914 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6833/3791366946,v1:172.21.15.179:6837/3791366946] up:boot 2024-06-03T01:25:14.914 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6834/262737514,v1:172.21.15.179:6838/262737514] up:boot 2024-06-03T01:25:14.914 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.179:6835/805745216,v1:172.21.15.179:6839/805745216] up:boot 2024-06-03T01:25:14.914 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6834/3126216501,v1:172.21.15.107:6835/3126216501] up:boot 2024-06-03T01:25:14.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6836/497887562,v1:172.21.15.107:6837/497887562] up:boot 2024-06-03T01:25:14.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.107:6838/3325408803,v1:172.21.15.107:6839/3325408803] up:boot 2024-06-03T01:25:14.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.94:6838/72315397,v1:172.21.15.94:6839/72315397] up:boot 2024-06-03T01:25:14.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: mds.? [v2:172.21.15.94:6840/4274631759,v1:172.21.15.94:6841/4274631759] up:boot 2024-06-03T01:25:14.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T01:25:14.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:25:14.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:25:14.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:14 smithi094 ceph-mon[32119]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:25:15.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: pgmap v466: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-03T01:25:15.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: osdmap e120: 12 total, 12 up, 12 in 2024-06-03T01:25:15.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6832/2719959718,v1:172.21.15.107:6833/2719959718] up:boot 2024-06-03T01:25:15.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6832/2644203537,v1:172.21.15.179:6836/2644203537] up:boot 2024-06-03T01:25:15.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6833/3791366946,v1:172.21.15.179:6837/3791366946] up:boot 2024-06-03T01:25:15.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6834/262737514,v1:172.21.15.179:6838/262737514] up:boot 2024-06-03T01:25:15.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.179:6835/805745216,v1:172.21.15.179:6839/805745216] up:boot 2024-06-03T01:25:15.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6834/3126216501,v1:172.21.15.107:6835/3126216501] up:boot 2024-06-03T01:25:15.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6836/497887562,v1:172.21.15.107:6837/497887562] up:boot 2024-06-03T01:25:15.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.107:6838/3325408803,v1:172.21.15.107:6839/3325408803] up:boot 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.94:6838/72315397,v1:172.21.15.94:6839/72315397] up:boot 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: mds.? [v2:172.21.15.94:6840/4274631759,v1:172.21.15.94:6841/4274631759] up:boot 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:25:15.035 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:25:15.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:25:15.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:25:15.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:25:15.036 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:14 smithi107 ceph-mon[36622]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:25:15.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: pgmap v466: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-06-03T01:25:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: osdmap e120: 12 total, 12 up, 12 in 2024-06-03T01:25:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6832/2719959718,v1:172.21.15.107:6833/2719959718] up:boot 2024-06-03T01:25:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6832/2644203537,v1:172.21.15.179:6836/2644203537] up:boot 2024-06-03T01:25:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6833/3791366946,v1:172.21.15.179:6837/3791366946] up:boot 2024-06-03T01:25:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6834/262737514,v1:172.21.15.179:6838/262737514] up:boot 2024-06-03T01:25:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.179:6835/805745216,v1:172.21.15.179:6839/805745216] up:boot 2024-06-03T01:25:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6834/3126216501,v1:172.21.15.107:6835/3126216501] up:boot 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6836/497887562,v1:172.21.15.107:6837/497887562] up:boot 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.107:6838/3325408803,v1:172.21.15.107:6839/3325408803] up:boot 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.94:6838/72315397,v1:172.21.15.94:6839/72315397] up:boot 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: mds.? [v2:172.21.15.94:6840/4274631759,v1:172.21.15.94:6841/4274631759] up:boot 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-03T01:25:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-03T01:25:15.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-03T01:25:15.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-03T01:25:15.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-03T01:25:15.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-03T01:25:15.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-03T01:25:15.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-03T01:25:15.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:14 smithi179 ceph-mon[37041]: from='mgr.15084 172.21.15.94:0/3645934195' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-03T01:25:16.746 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:16 smithi094 ceph-mon[32119]: pgmap v469: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-03T01:25:16.783 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:16 smithi107 ceph-mon[36622]: pgmap v469: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-03T01:25:17.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:16 smithi179 ceph-mon[37041]: pgmap v469: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-03T01:25:17.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 -- 172.21.15.94:0/3376806458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c8074860 msgr2=0x7f81c8074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 --2- 172.21.15.94:0/3376806458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c8074860 0x7f81c8074c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009990 tx=0x7f81bc009ca0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 -- 172.21.15.94:0/3376806458 shutdown_connections 2024-06-03T01:25:17.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 --2- 172.21.15.94:0/3376806458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c810ec40 0x7f81c8111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 --2- 172.21.15.94:0/3376806458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c8074860 0x7f81c8074c40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 --2- 172.21.15.94:0/3376806458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81c8075180 0x7f81c810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 -- 172.21.15.94:0/3376806458 >> 172.21.15.94:0/3376806458 conn(0x7f81c80fdeb0 msgr2=0x7f81c81002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:17.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 -- 172.21.15.94:0/3376806458 shutdown_connections 2024-06-03T01:25:17.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.329+0000 7f81ce200700 1 -- 172.21.15.94:0/3376806458 wait complete. 2024-06-03T01:25:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.330+0000 7f81ce200700 1 Processor -- start 2024-06-03T01:25:17.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81ce200700 1 -- start start 2024-06-03T01:25:17.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81ce200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c8074860 0x7f81c819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:17.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81ce200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81c8075180 0x7f81c819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:17.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81ce200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c810ec40 0x7f81c819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:17.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81ce200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8114940 con 0x7f81c810ec40 2024-06-03T01:25:17.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81ce200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81147c0 con 0x7f81c8074860 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81ce200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8114ac0 con 0x7f81c8075180 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81cce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c8074860 0x7f81c819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81cce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c8074860 0x7f81c819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:36548/0 (socket says 172.21.15.94:36548) 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81cce00700 1 -- 172.21.15.94:0/758661102 learned_addr learned my addr 172.21.15.94:0/758661102 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81c7e00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81c8075180 0x7f81c819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.331+0000 7f81cd800700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c810ec40 0x7f81c819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81cce00700 1 -- 172.21.15.94:0/758661102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81c8075180 msgr2=0x7f81c819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81cce00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81c8075180 0x7f81c819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81cce00700 1 -- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c810ec40 msgr2=0x7f81c819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81cce00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c810ec40 0x7f81c819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81cce00700 1 -- 172.21.15.94:0/758661102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c819fdb0 con 0x7f81c8074860 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81cd800700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c810ec40 0x7f81c819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81cce00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c8074860 0x7f81c819adf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f81bc000c00 tx=0x7f81bc00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81c5600700 1 -- 172.21.15.94:0/758661102 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc027070 con 0x7f81c8074860 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81c5600700 1 -- 172.21.15.94:0/758661102 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81bc003d70 con 0x7f81c8074860 2024-06-03T01:25:17.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81ce200700 1 -- 172.21.15.94:0/758661102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c81a0040 con 0x7f81c8074860 2024-06-03T01:25:17.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.332+0000 7f81c5600700 1 -- 172.21.15.94:0/758661102 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00f460 con 0x7f81c8074860 2024-06-03T01:25:17.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.333+0000 7f81ce200700 1 -- 172.21.15.94:0/758661102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81a7920 con 0x7f81c8074860 2024-06-03T01:25:17.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.334+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c8069180 con 0x7f81c8074860 2024-06-03T01:25:17.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.336+0000 7f81c5600700 1 -- 172.21.15.94:0/758661102 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f81bc02a070 con 0x7f81c8074860 2024-06-03T01:25:17.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.337+0000 7f81c5600700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f81b00a2d90 0x7f81b00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:17.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.337+0000 7f81c5600700 1 -- 172.21.15.94:0/758661102 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9905+0+0 (secure 0 0 0) 0x7f81bc0d0ea0 con 0x7f81c8074860 2024-06-03T01:25:17.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.337+0000 7f81c7e00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f81b00a2d90 0x7f81b00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:17.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.338+0000 7f81c7e00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f81b00a2d90 0x7f81b00a5250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81c8101ab0 tx=0x7f81b4009260 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:17.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.340+0000 7f81c5600700 1 -- 172.21.15.94:0/758661102 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f81bc013070 con 0x7f81c8074860 2024-06-03T01:25:17.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.608+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f81c80630a0 con 0x7f81c8074860 2024-06-03T01:25:17.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.660+0000 7f81c5600700 1 -- 172.21.15.94:0/758661102 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7f81bc09d580 con 0x7f81c8074860 2024-06-03T01:25:17.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f81b00a2d90 msgr2=0x7f81b00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f81b00a2d90 0x7f81b00a5250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81c8101ab0 tx=0x7f81b4009260 comp rx=0 tx=0).stop 2024-06-03T01:25:17.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c8074860 msgr2=0x7f81c819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:17.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c8074860 0x7f81c819adf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f81bc000c00 tx=0x7f81bc00bab0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 shutdown_connections 2024-06-03T01:25:17.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f81b00a2d90 0x7f81b00a5250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c8074860 0x7f81c819adf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81c810ec40 0x7f81c819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 --2- 172.21.15.94:0/758661102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81c8075180 0x7f81c819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 >> 172.21.15.94:0/758661102 conn(0x7f81c80fdeb0 msgr2=0x7f81c80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:17.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 shutdown_connections 2024-06-03T01:25:17.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:17.664+0000 7f81baa00700 1 -- 172.21.15.94:0/758661102 wait complete. 2024-06-03T01:25:18.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:17 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/758661102' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:25:18.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:17 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:25:18.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:17 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/758661102' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:25:18.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:17 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:25:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:17 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/758661102' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:25:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:17 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-03T01:25:18.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-03T01:25:18.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:18 smithi094 ceph-mon[32119]: pgmap v470: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 135 B/s wr, 8 op/s 2024-06-03T01:25:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:18 smithi094 ceph-mon[32119]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T01:25:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:18 smithi094 ceph-mon[32119]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T01:25:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:18 smithi094 ceph-mon[32119]: Cluster is now healthy 2024-06-03T01:25:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:18 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T01:25:18.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:18 smithi094 ceph-mon[32119]: fsmap 12 up:standby 2024-06-03T01:25:19.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:18 smithi107 ceph-mon[36622]: pgmap v470: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 135 B/s wr, 8 op/s 2024-06-03T01:25:19.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:18 smithi107 ceph-mon[36622]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T01:25:19.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:18 smithi107 ceph-mon[36622]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T01:25:19.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:18 smithi107 ceph-mon[36622]: Cluster is now healthy 2024-06-03T01:25:19.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:18 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T01:25:19.034 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:18 smithi107 ceph-mon[36622]: fsmap 12 up:standby 2024-06-03T01:25:19.089 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:25:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:18 smithi179 ceph-mon[37041]: pgmap v470: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 135 B/s wr, 8 op/s 2024-06-03T01:25:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:18 smithi179 ceph-mon[37041]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-03T01:25:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:18 smithi179 ceph-mon[37041]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-03T01:25:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:18 smithi179 ceph-mon[37041]: Cluster is now healthy 2024-06-03T01:25:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:18 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-03T01:25:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:18 smithi179 ceph-mon[37041]: fsmap 12 up:standby 2024-06-03T01:25:20.925 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:20 smithi094 ceph-mon[32119]: pgmap v471: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 127 B/s wr, 7 op/s 2024-06-03T01:25:21.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:20 smithi107 ceph-mon[36622]: pgmap v471: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 127 B/s wr, 7 op/s 2024-06-03T01:25:21.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:20 smithi179 ceph-mon[37041]: pgmap v471: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 127 B/s wr, 7 op/s 2024-06-03T01:25:21.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.545+0000 7fa14fa00700 1 -- 172.21.15.94:0/1059514235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14806aa30 msgr2=0x7fa14806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.545+0000 7fa14fa00700 1 --2- 172.21.15.94:0/1059514235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14806aa30 0x7fa14806ae10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa138009990 tx=0x7fa138009ca0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.545+0000 7fa14fa00700 1 -- 172.21.15.94:0/1059514235 shutdown_connections 2024-06-03T01:25:21.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.545+0000 7fa14fa00700 1 --2- 172.21.15.94:0/1059514235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14806aa30 0x7fa14806ae10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.545+0000 7fa14fa00700 1 --2- 172.21.15.94:0/1059514235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14810ef30 0x7fa148111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.545+0000 7fa14fa00700 1 --2- 172.21.15.94:0/1059514235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14806b350 0x7fa14810e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.545+0000 7fa14fa00700 1 -- 172.21.15.94:0/1059514235 >> 172.21.15.94:0/1059514235 conn(0x7fa148078300 msgr2=0x7fa148078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:21.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.546+0000 7fa14fa00700 1 -- 172.21.15.94:0/1059514235 shutdown_connections 2024-06-03T01:25:21.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.546+0000 7fa14fa00700 1 -- 172.21.15.94:0/1059514235 wait complete. 2024-06-03T01:25:21.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 Processor -- start 2024-06-03T01:25:21.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 -- start start 2024-06-03T01:25:21.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14806aa30 0x7fa14819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14806b350 0x7fa14819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14810ef30 0x7fa14819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481149c0 con 0x7fa14806aa30 2024-06-03T01:25:21.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa148114840 con 0x7fa14810ef30 2024-06-03T01:25:21.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.547+0000 7fa14fa00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa148114b40 con 0x7fa14806b350 2024-06-03T01:25:21.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14806aa30 0x7fa14819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14806aa30 0x7fa14819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59834/0 (socket says 172.21.15.94:59834) 2024-06-03T01:25:21.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 -- 172.21.15.94:0/3426826873 learned_addr learned my addr 172.21.15.94:0/3426826873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:21.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa13fe00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14806b350 0x7fa14819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14da00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14810ef30 0x7fa14819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 -- 172.21.15.94:0/3426826873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14806b350 msgr2=0x7fa14819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14806b350 0x7fa14819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 -- 172.21.15.94:0/3426826873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14810ef30 msgr2=0x7fa14819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:21.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14810ef30 0x7fa14819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:21.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 -- 172.21.15.94:0/3426826873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14819fe90 con 0x7fa14806aa30 2024-06-03T01:25:21.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.548+0000 7fa14d000700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14806aa30 0x7fa14819aed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b820 tx=0x7fa13800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.549+0000 7fa146000700 1 -- 172.21.15.94:0/3426826873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138027070 con 0x7fa14806aa30 2024-06-03T01:25:21.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.549+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1481a0120 con 0x7fa14806aa30 2024-06-03T01:25:21.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.549+0000 7fa146000700 1 -- 172.21.15.94:0/3426826873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1380056d0 con 0x7fa14806aa30 2024-06-03T01:25:21.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.549+0000 7fa146000700 1 -- 172.21.15.94:0/3426826873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138022e00 con 0x7fa14806aa30 2024-06-03T01:25:21.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.549+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a7a00 con 0x7fa14806aa30 2024-06-03T01:25:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.550+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148069180 con 0x7fa14806aa30 2024-06-03T01:25:21.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.552+0000 7fa146000700 1 -- 172.21.15.94:0/3426826873 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa13802a070 con 0x7fa14806aa30 2024-06-03T01:25:21.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.553+0000 7fa146000700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa12c0a2d90 0x7fa12c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:21.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.553+0000 7fa146000700 1 -- 172.21.15.94:0/3426826873 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9905+0+0 (secure 0 0 0) 0x7fa1380d1320 con 0x7fa14806aa30 2024-06-03T01:25:21.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.554+0000 7fa13fe00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa12c0a2d90 0x7fa12c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:21.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.554+0000 7fa13fe00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa12c0a2d90 0x7fa12c0a5250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1481028c0 tx=0x7fa134009260 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:21.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.556+0000 7fa146000700 1 -- 172.21.15.94:0/3426826873 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa138031070 con 0x7fa14806aa30 2024-06-03T01:25:21.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:21.782+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa14819c1e0 con 0x7fa14806aa30 2024-06-03T01:25:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.667+0000 7fa146000700 1 -- 172.21.15.94:0/3426826873 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) ==== 181+0+0 (secure 0 0 0) 0x7fa138013070 con 0x7fa14806aa30 2024-06-03T01:25:22.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.672+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa12c0a2d90 msgr2=0x7fa12c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.672+0000 7fa14fa00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa12c0a2d90 0x7fa12c0a5250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1481028c0 tx=0x7fa134009260 comp rx=0 tx=0).stop 2024-06-03T01:25:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.672+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14806aa30 msgr2=0x7fa14819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.672+0000 7fa14fa00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14806aa30 0x7fa14819aed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b820 tx=0x7fa13800bab0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.673+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 shutdown_connections 2024-06-03T01:25:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.673+0000 7fa14fa00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fa12c0a2d90 0x7fa12c0a5250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.673+0000 7fa14fa00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa14810ef30 0x7fa14819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.673+0000 7fa14fa00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa14806aa30 0x7fa14819aed0 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b820 tx=0x7fa13800bab0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.673+0000 7fa14fa00700 1 --2- 172.21.15.94:0/3426826873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa14806b350 0x7fa14819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.673+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 >> 172.21.15.94:0/3426826873 conn(0x7fa148078300 msgr2=0x7fa148110380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.675+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 shutdown_connections 2024-06-03T01:25:22.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:22.675+0000 7fa14fa00700 1 -- 172.21.15.94:0/3426826873 wait complete. 2024-06-03T01:25:22.678 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-06-03T01:25:23.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:22 smithi107 ceph-mon[36622]: pgmap v472: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-06-03T01:25:23.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:22 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3426826873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:23.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:22 smithi094 ceph-mon[32119]: pgmap v472: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-06-03T01:25:23.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:22 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3426826873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:23.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:22 smithi179 ceph-mon[37041]: pgmap v472: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-06-03T01:25:23.139 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:22 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3426826873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:23.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-03T01:25:23.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:23 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/3426826873' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:23.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:23 smithi094 ceph-mon[32119]: osdmap e121: 12 total, 12 up, 12 in 2024-06-03T01:25:24.006 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:23 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/3426826873' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:24.007 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:23 smithi179 ceph-mon[37041]: osdmap e121: 12 total, 12 up, 12 in 2024-06-03T01:25:24.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:23 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/3426826873' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:24.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:23 smithi107 ceph-mon[36622]: osdmap e121: 12 total, 12 up, 12 in 2024-06-03T01:25:24.046 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:25:24.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:24 smithi094 ceph-mon[32119]: pgmap v474: 97 pgs: 97 active+clean; 256 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:25.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:24 smithi107 ceph-mon[36622]: pgmap v474: 97 pgs: 97 active+clean; 256 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:25.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:24 smithi179 ceph-mon[37041]: pgmap v474: 97 pgs: 97 active+clean; 256 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:26.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 -- 172.21.15.94:0/116186429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f00746c0 msgr2=0x7f51f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:26.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 --2- 172.21.15.94:0/116186429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f00746c0 0x7f51f0074aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f51e40098f0 tx=0x7f51e4009c00 comp rx=0 tx=0).stop 2024-06-03T01:25:26.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 -- 172.21.15.94:0/116186429 shutdown_connections 2024-06-03T01:25:26.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 --2- 172.21.15.94:0/116186429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51f010ee90 0x7f51f0111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:26.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 --2- 172.21.15.94:0/116186429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f00746c0 0x7f51f0074aa0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:26.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 --2- 172.21.15.94:0/116186429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f0074fe0 0x7f51f010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:26.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 -- 172.21.15.94:0/116186429 >> 172.21.15.94:0/116186429 conn(0x7f51f00795f0 msgr2=0x7f51f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:26.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 -- 172.21.15.94:0/116186429 shutdown_connections 2024-06-03T01:25:26.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.617+0000 7f51f8400700 1 -- 172.21.15.94:0/116186429 wait complete. 2024-06-03T01:25:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.618+0000 7f51f8400700 1 Processor -- start 2024-06-03T01:25:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.618+0000 7f51f8400700 1 -- start start 2024-06-03T01:25:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f8400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f00746c0 0x7f51f019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f8400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f0074fe0 0x7f51f019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f8400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51f010ee90 0x7f51f019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f8400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0114990 con 0x7f51f010ee90 2024-06-03T01:25:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f8400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0114810 con 0x7f51f00746c0 2024-06-03T01:25:26.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f8400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0114b10 con 0x7f51f0074fe0 2024-06-03T01:25:26.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f5a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f00746c0 0x7f51f019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:26.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f5000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f0074fe0 0x7f51f019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:26.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f5a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f00746c0 0x7f51f019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.94:48694/0 (socket says 172.21.15.94:48694) 2024-06-03T01:25:26.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.619+0000 7f51f5a00700 1 -- 172.21.15.94:0/1305723122 learned_addr learned my addr 172.21.15.94:0/1305723122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:26.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f6400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51f010ee90 0x7f51f019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:26.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f5a00700 1 -- 172.21.15.94:0/1305723122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f0074fe0 msgr2=0x7f51f019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f5a00700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f0074fe0 0x7f51f019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f5a00700 1 -- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51f010ee90 msgr2=0x7f51f019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f5a00700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51f010ee90 0x7f51f019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f5a00700 1 -- 172.21.15.94:0/1305723122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f019fe20 con 0x7f51f00746c0 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f6400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51f010ee90 0x7f51f019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.620+0000 7f51f5a00700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f00746c0 0x7f51f019ae60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009230 tx=0x7f51e400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.621+0000 7f51ee000700 1 -- 172.21.15.94:0/1305723122 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4005200 con 0x7f51f00746c0 2024-06-03T01:25:26.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.621+0000 7f51ee000700 1 -- 172.21.15.94:0/1305723122 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e400b560 con 0x7f51f00746c0 2024-06-03T01:25:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.621+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f01a00b0 con 0x7f51f00746c0 2024-06-03T01:25:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.621+0000 7f51ee000700 1 -- 172.21.15.94:0/1305723122 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4003b10 con 0x7f51f00746c0 2024-06-03T01:25:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.621+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f01a7990 con 0x7f51f00746c0 2024-06-03T01:25:26.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.622+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f0075db0 con 0x7f51f00746c0 2024-06-03T01:25:26.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.624+0000 7f51ee000700 1 -- 172.21.15.94:0/1305723122 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f51e4029060 con 0x7f51f00746c0 2024-06-03T01:25:26.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.625+0000 7f51ee000700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f51d80a2d90 0x7f51d80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:26.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.625+0000 7f51ee000700 1 -- 172.21.15.94:0/1305723122 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9321+0+0 (secure 0 0 0) 0x7f51e40d03a0 con 0x7f51f00746c0 2024-06-03T01:25:26.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.626+0000 7f51f5000700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f51d80a2d90 0x7f51d80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:26.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.626+0000 7f51f5000700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f51d80a2d90 0x7f51d80a5250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009680 tx=0x7f51dc009260 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:26.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.628+0000 7f51ee000700 1 -- 172.21.15.94:0/1305723122 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f51e4013030 con 0x7f51f00746c0 2024-06-03T01:25:26.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:26 smithi094 ceph-mon[32119]: pgmap v475: 97 pgs: 97 active+clean; 256 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:26.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:26.906+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f51f019c170 con 0x7f51f00746c0 2024-06-03T01:25:27.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:26 smithi107 ceph-mon[36622]: pgmap v475: 97 pgs: 97 active+clean; 256 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:27.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:26 smithi179 ceph-mon[37041]: pgmap v475: 97 pgs: 97 active+clean; 256 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:27.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.711+0000 7f51ee000700 1 -- 172.21.15.94:0/1305723122 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) ==== 169+0+0 (secure 0 0 0) 0x7f51e409ccd0 con 0x7f51f00746c0 2024-06-03T01:25:27.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.715+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f51d80a2d90 msgr2=0x7f51d80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:27.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.715+0000 7f51f8400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f51d80a2d90 0x7f51d80a5250 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009680 tx=0x7f51dc009260 comp rx=0 tx=0).stop 2024-06-03T01:25:27.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.715+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f00746c0 msgr2=0x7f51f019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:27.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.715+0000 7f51f8400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f00746c0 0x7f51f019ae60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009230 tx=0x7f51e400fa70 comp rx=0 tx=0).stop 2024-06-03T01:25:27.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.716+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 shutdown_connections 2024-06-03T01:25:27.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.716+0000 7f51f8400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7f51d80a2d90 0x7f51d80a5250 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009680 tx=0x7f51dc009260 comp rx=0 tx=0).stop 2024-06-03T01:25:27.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.716+0000 7f51f8400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f00746c0 0x7f51f019ae60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:27.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.716+0000 7f51f8400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51f010ee90 0x7f51f019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:27.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.716+0000 7f51f8400700 1 --2- 172.21.15.94:0/1305723122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f0074fe0 0x7f51f019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:27.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.716+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 >> 172.21.15.94:0/1305723122 conn(0x7f51f00795f0 msgr2=0x7f51f00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:27.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.716+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 shutdown_connections 2024-06-03T01:25:27.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:27.717+0000 7f51f8400700 1 -- 172.21.15.94:0/1305723122 wait complete. 2024-06-03T01:25:27.719 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-06-03T01:25:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:27 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/1305723122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:27.975 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:27 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:28.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:27 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/1305723122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:28.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:27 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:27 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/1305723122' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:27 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:28.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-03T01:25:29.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:28 smithi107 ceph-mon[36622]: pgmap v476: 97 pgs: 97 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:29.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:28 smithi107 ceph-mon[36622]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:29.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:28 smithi107 ceph-mon[36622]: osdmap e122: 12 total, 12 up, 12 in 2024-06-03T01:25:29.098 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:28 smithi094 ceph-mon[32119]: pgmap v476: 97 pgs: 97 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:29.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:28 smithi094 ceph-mon[32119]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:29.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:28 smithi094 ceph-mon[32119]: osdmap e122: 12 total, 12 up, 12 in 2024-06-03T01:25:29.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:28 smithi179 ceph-mon[37041]: pgmap v476: 97 pgs: 97 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:29.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:28 smithi179 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:29.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:28 smithi179 ceph-mon[37041]: osdmap e122: 12 total, 12 up, 12 in 2024-06-03T01:25:29.240 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/mon.a/config 2024-06-03T01:25:30.033 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:29 smithi107 ceph-mon[36622]: pgmap v478: 33 pgs: 33 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:30.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:29 smithi094 ceph-mon[32119]: pgmap v478: 33 pgs: 33 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:29 smithi179 ceph-mon[37041]: pgmap v478: 33 pgs: 33 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.612+0000 7fb2ec200700 1 -- 172.21.15.94:0/4098641966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4107840 msgr2=0x7fb2e410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.612+0000 7fb2ec200700 1 --2- 172.21.15.94:0/4098641966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4107840 0x7fb2e410f3c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0099c0 tx=0x7fb2dc009cd0 comp rx=0 tx=0).stop 2024-06-03T01:25:31.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.613+0000 7fb2ec200700 1 -- 172.21.15.94:0/4098641966 shutdown_connections 2024-06-03T01:25:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.613+0000 7fb2ec200700 1 --2- 172.21.15.94:0/4098641966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e4102740 0x7fb2e4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.613+0000 7fb2ec200700 1 --2- 172.21.15.94:0/4098641966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4107840 0x7fb2e410f3c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.613+0000 7fb2ec200700 1 --2- 172.21.15.94:0/4098641966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e41030f0 0x7fb2e4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.613+0000 7fb2ec200700 1 -- 172.21.15.94:0/4098641966 >> 172.21.15.94:0/4098641966 conn(0x7fb2e40795f0 msgr2=0x7fb2e4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.613+0000 7fb2ec200700 1 -- 172.21.15.94:0/4098641966 shutdown_connections 2024-06-03T01:25:31.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.613+0000 7fb2ec200700 1 -- 172.21.15.94:0/4098641966 wait complete. 2024-06-03T01:25:31.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.614+0000 7fb2ec200700 1 Processor -- start 2024-06-03T01:25:31.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.614+0000 7fb2ec200700 1 -- start start 2024-06-03T01:25:31.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2ec200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4102740 0x7fb2e419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:31.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2ec200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e41030f0 0x7fb2e419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:31.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2ec200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e4107840 0x7fb2e41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:31.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2ec200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4112760 con 0x7fb2e4102740 2024-06-03T01:25:31.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2ec200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41125e0 con 0x7fb2e4107840 2024-06-03T01:25:31.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2ec200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41128e0 con 0x7fb2e41030f0 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2e9800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4102740 0x7fb2e419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2e9800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4102740 0x7fb2e419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33664/0 (socket says 172.21.15.94:33664) 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2e9800700 1 -- 172.21.15.94:0/2703881915 learned_addr learned my addr 172.21.15.94:0/2703881915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2e8e00700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e41030f0 0x7fb2e419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.615+0000 7fb2ea200700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e4107840 0x7fb2e41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e9800700 1 -- 172.21.15.94:0/2703881915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e41030f0 msgr2=0x7fb2e419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e9800700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e41030f0 0x7fb2e419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:31.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e9800700 1 -- 172.21.15.94:0/2703881915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e4107840 msgr2=0x7fb2e41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e9800700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e4107840 0x7fb2e41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e9800700 1 -- 172.21.15.94:0/2703881915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e4199120 con 0x7fb2e4102740 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e9800700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4102740 0x7fb2e419e7b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800ef10 tx=0x7fb2d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e2000700 1 -- 172.21.15.94:0/2703881915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800c9c0 con 0x7fb2e4102740 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e2000700 1 -- 172.21.15.94:0/2703881915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d8019070 con 0x7fb2e4102740 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2e2000700 1 -- 172.21.15.94:0/2703881915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8007e60 con 0x7fb2e4102740 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e41993b0 con 0x7fb2e4102740 2024-06-03T01:25:31.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.616+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a5680 con 0x7fb2e4102740 2024-06-03T01:25:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.618+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e4069180 con 0x7fb2e4102740 2024-06-03T01:25:31.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.620+0000 7fb2e2000700 1 -- 172.21.15.94:0/2703881915 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb2d8012050 con 0x7fb2e4102740 2024-06-03T01:25:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.621+0000 7fb2e2000700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb2cc0a2e10 0x7fb2cc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-03T01:25:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.621+0000 7fb2e2000700 1 -- 172.21.15.94:0/2703881915 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8733+0+0 (secure 0 0 0) 0x7fb2d80cf490 con 0x7fb2e4102740 2024-06-03T01:25:31.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.621+0000 7fb2e8e00700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb2cc0a2e10 0x7fb2cc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-03T01:25:31.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.622+0000 7fb2e8e00700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb2cc0a2e10 0x7fb2cc0a52d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb2e41070c0 tx=0x7fb2d000b410 comp rx=0 tx=0).ready entity=mgr.15084 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-03T01:25:31.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.626+0000 7fb2e2000700 1 -- 172.21.15.94:0/2703881915 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb2d8010040 con 0x7fb2e4102740 2024-06-03T01:25:31.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:31.847+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb2e402ddc0 con 0x7fb2e4102740 2024-06-03T01:25:32.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.221+0000 7fb2e2000700 1 -- 172.21.15.94:0/2703881915 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v123) ==== 178+0+0 (secure 0 0 0) 0x7fb2d809bf80 con 0x7fb2e4102740 2024-06-03T01:25:32.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.228+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb2cc0a2e10 msgr2=0x7fb2cc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:32.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.228+0000 7fb2ec200700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb2cc0a2e10 0x7fb2cc0a52d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb2e41070c0 tx=0x7fb2d000b410 comp rx=0 tx=0).stop 2024-06-03T01:25:32.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.228+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4102740 msgr2=0x7fb2e419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-03T01:25:32.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.228+0000 7fb2ec200700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4102740 0x7fb2e419e7b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800ef10 tx=0x7fb2d800c3a0 comp rx=0 tx=0).stop 2024-06-03T01:25:32.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.229+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 shutdown_connections 2024-06-03T01:25:32.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.229+0000 7fb2ec200700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:6800/2692604051,v1:172.21.15.94:6801/2692604051] conn(0x7fb2cc0a2e10 0x7fb2cc0a52d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.229+0000 7fb2ec200700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e4107840 0x7fb2e41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.229+0000 7fb2ec200700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2e4102740 0x7fb2e419e7b0 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800ef10 tx=0x7fb2d800c3a0 comp rx=0 tx=0).stop 2024-06-03T01:25:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.229+0000 7fb2ec200700 1 --2- 172.21.15.94:0/2703881915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e41030f0 0x7fb2e419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-03T01:25:32.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.229+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 >> 172.21.15.94:0/2703881915 conn(0x7fb2e40795f0 msgr2=0x7fb2e4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-03T01:25:32.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.231+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 shutdown_connections 2024-06-03T01:25:32.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-06-03T01:25:32.231+0000 7fb2ec200700 1 -- 172.21.15.94:0/2703881915 wait complete. 2024-06-03T01:25:32.235 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data_ec' removed 2024-06-03T01:25:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:32 smithi107 ceph-mon[36622]: pgmap v479: 33 pgs: 33 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:32.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:32 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2703881915' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:32.580 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:32 smithi094 ceph-mon[32119]: pgmap v479: 33 pgs: 33 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:32.581 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:32 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2703881915' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:32 smithi179 ceph-mon[37041]: pgmap v479: 33 pgs: 33 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-03T01:25:32.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:32 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2703881915' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-03T01:25:33.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:25:33.210 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:25:33.245 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:25:33.277 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-03T01:25:33.277 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T01:25:33.328 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T01:25:33.346 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-03T01:25:33.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:33 smithi094 ceph-mon[32119]: from='client.? 172.21.15.94:0/2703881915' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:33.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:33 smithi094 ceph-mon[32119]: osdmap e123: 12 total, 12 up, 12 in 2024-06-03T01:25:33.363 INFO:tasks.cephadm:Stopping all daemons... 2024-06-03T01:25:33.364 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-03T01:25:33.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a 2024-06-03T01:25:33.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:33 smithi107 ceph-mon[36622]: from='client.? 172.21.15.94:0/2703881915' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:33.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:33 smithi107 ceph-mon[36622]: osdmap e123: 12 total, 12 up, 12 in 2024-06-03T01:25:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:33 smithi179 ceph-mon[37041]: from='client.? 172.21.15.94:0/2703881915' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-03T01:25:33.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:33 smithi179 ceph-mon[37041]: osdmap e123: 12 total, 12 up, 12 in 2024-06-03T01:25:33.849 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:33 smithi094 systemd[1]: Stopping Ceph mon.a for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:34.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a[32115]: 2024-06-03T01:25:33.920+0000 7fc8d7c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:34.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a[32115]: 2024-06-03T01:25:33.920+0000 7fc8d7c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-03T01:25:34.350 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:34 smithi094 podman[102675]: 2024-06-03 01:25:34.096645654 +0000 UTC m=+0.339030059 container died a6a853b7a1fe6e4bd37b84b1e5232351ffd065385417ce80dd4bd9969b82c816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-03T01:25:34.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:34 smithi094 podman[102675]: 2024-06-03 01:25:34.572791237 +0000 UTC m=+0.815175697 container cleanup a6a853b7a1fe6e4bd37b84b1e5232351ffd065385417ce80dd4bd9969b82c816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-06-03T01:25:34.850 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:34 smithi094 bash[102675]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a 2024-06-03T01:25:35.349 INFO:journalctl@ceph.mon.a.smithi094.stdout:Jun 03 01:25:34 smithi094 podman[102688]: 2024-06-03 01:25:34.849317765 +0000 UTC m=+0.730445763 container remove a6a853b7a1fe6e4bd37b84b1e5232351ffd065385417ce80dd4bd9969b82c816 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-a, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-03T01:25:35.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-03T01:25:35.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:25:35.544 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-03T01:25:35.544 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-03T01:25:35.544 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.b 2024-06-03T01:25:35.881 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:35 smithi107 systemd[1]: Stopping Ceph mon.b for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:35 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-b[36618]: 2024-06-03T01:25:35.880+0000 7f7531f94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:35 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-b[36618]: 2024-06-03T01:25:35.880+0000 7f7531f94700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-03T01:25:36.283 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:36 smithi107 podman[74805]: 2024-06-03 01:25:36.075587289 +0000 UTC m=+0.324616747 container died 62de82cd24ed69ae7a4739efee425b61d72fb84745fcc4340bb62eaf5ce34379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-03T01:25:36.567 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:36 smithi107 podman[74805]: 2024-06-03 01:25:36.498948712 +0000 UTC m=+0.747978168 container cleanup 62de82cd24ed69ae7a4739efee425b61d72fb84745fcc4340bb62eaf5ce34379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-03T01:25:36.568 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:36 smithi107 bash[74805]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-b 2024-06-03T01:25:36.982 INFO:journalctl@ceph.mon.b.smithi107.stdout:Jun 03 01:25:36 smithi107 podman[74817]: 2024-06-03 01:25:36.759397544 +0000 UTC m=+0.684550997 container remove 62de82cd24ed69ae7a4739efee425b61d72fb84745fcc4340bb62eaf5ce34379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-03T01:25:36.996 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-03T01:25:37.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:25:37.054 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-03T01:25:37.054 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-03T01:25:37.054 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.c 2024-06-03T01:25:37.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:37 smithi179 systemd[1]: Stopping Ceph mon.c for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-c[37037]: 2024-06-03T01:25:37.373+0000 7f0bd81b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:37.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-c[37037]: 2024-06-03T01:25:37.374+0000 7f0bd81b2700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-03T01:25:37.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:37 smithi179 podman[75115]: 2024-06-03 01:25:37.569287573 +0000 UTC m=+0.320736627 container died a77623c6a7daa68cc507f22ae1abe2a6ae236f42bcccb6690812540964964db7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-03T01:25:38.234 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:37 smithi179 podman[75115]: 2024-06-03 01:25:37.967150456 +0000 UTC m=+0.718599509 container cleanup a77623c6a7daa68cc507f22ae1abe2a6ae236f42bcccb6690812540964964db7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-03T01:25:38.235 INFO:journalctl@ceph.mon.c.smithi179.stdout:Jun 03 01:25:37 smithi179 bash[75115]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mon-c 2024-06-03T01:25:38.391 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-03T01:25:38.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:25:38.432 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-03T01:25:38.432 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-03T01:25:38.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.x 2024-06-03T01:25:38.755 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:25:38 smithi094 systemd[1]: Stopping Ceph mgr.x for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:39.066 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:25:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:25:39.002+0000 7f0522200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:39.067 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:25:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x[32359]: 2024-06-03T01:25:39.002+0000 7f0522200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-03T01:25:39.349 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:25:39 smithi094 podman[102795]: 2024-06-03 01:25:39.098173129 +0000 UTC m=+0.249965369 container died 0dcafd448720e14b35b84e142492819c43f3f6b5408ffbc7c94ccd0d4c32d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-03T01:25:39.849 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:25:39 smithi094 podman[102795]: 2024-06-03 01:25:39.62068348 +0000 UTC m=+0.772475693 container cleanup 0dcafd448720e14b35b84e142492819c43f3f6b5408ffbc7c94ccd0d4c32d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-03T01:25:39.850 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:25:39 smithi094 bash[102795]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x 2024-06-03T01:25:40.300 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Jun 03 01:25:39 smithi094 podman[102815]: 2024-06-03 01:25:39.869293304 +0000 UTC m=+0.762813508 container remove 0dcafd448720e14b35b84e142492819c43f3f6b5408ffbc7c94ccd0d4c32d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-03T01:25:40.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-03T01:25:40.453 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:25:40.454 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-03T01:25:40.454 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-03T01:25:40.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.y 2024-06-03T01:25:40.783 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:25:40 smithi107 systemd[1]: Stopping Ceph mgr.y for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:41.147 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:25:40 smithi107 podman[74940]: 2024-06-03 01:25:40.826176645 +0000 UTC m=+0.162908867 container died 726312a8f79e95d23024c6a0ffd4018c63b70a64c475c054b800ba87f95c7a47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:25:41.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:25:41 smithi107 podman[74940]: 2024-06-03 01:25:41.254450437 +0000 UTC m=+0.591182657 container cleanup 726312a8f79e95d23024c6a0ffd4018c63b70a64c475c054b800ba87f95c7a47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-03T01:25:41.533 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:25:41 smithi107 bash[74940]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y 2024-06-03T01:25:41.828 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:25:41 smithi107 podman[74953]: 2024-06-03 01:25:41.573893348 +0000 UTC m=+0.748417048 container remove 726312a8f79e95d23024c6a0ffd4018c63b70a64c475c054b800ba87f95c7a47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-03T01:25:41.828 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Jun 03 01:25:41 smithi107 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-03T01:25:41.832 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-03T01:25:41.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:25:41.867 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-03T01:25:41.867 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-03T01:25:41.868 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.z 2024-06-03T01:25:42.192 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:25:41 smithi179 systemd[1]: Stopping Ceph mgr.z for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:42.472 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:25:42 smithi179 podman[75241]: 2024-06-03 01:25:42.192815563 +0000 UTC m=+0.134122582 container died 729cadb4f3655de127a7150f12b79f3b87e884af190758dbe314ffade2245e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-03T01:25:42.832 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:25:42 smithi179 podman[75241]: 2024-06-03 01:25:42.581095432 +0000 UTC m=+0.522402442 container cleanup 729cadb4f3655de127a7150f12b79f3b87e884af190758dbe314ffade2245e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-03T01:25:42.832 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Jun 03 01:25:42 smithi179 bash[75241]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mgr-z 2024-06-03T01:25:42.999 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-03T01:25:43.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:25:43.032 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-03T01:25:43.032 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-03T01:25:43.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.0 2024-06-03T01:25:43.599 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:43 smithi094 systemd[1]: Stopping Ceph osd.0 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:44.099 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0[47187]: 2024-06-03T01:25:43.635+0000 7f0c85e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:44.100 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0[47187]: 2024-06-03T01:25:43.635+0000 7f0c85e00700 -1 osd.0 123 *** Got signal Terminated *** 2024-06-03T01:25:44.100 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0[47187]: 2024-06-03T01:25:43.635+0000 7f0c85e00700 -1 osd.0 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:25:49.438 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:49 smithi094 podman[102924]: 2024-06-03 01:25:49.181032472 +0000 UTC m=+5.704516002 container died d4caab23f6a94101bcb47aad0bfbf7ed15d6673017bb0a433ff429f3b5bd496b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-03T01:25:49.964 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:49 smithi094 podman[102924]: 2024-06-03 01:25:49.702475857 +0000 UTC m=+6.225959424 container cleanup d4caab23f6a94101bcb47aad0bfbf7ed15d6673017bb0a433ff429f3b5bd496b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:25:49.964 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:49 smithi094 bash[102924]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0 2024-06-03T01:25:50.349 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:49 smithi094 podman[102942]: 2024-06-03 01:25:49.962395355 +0000 UTC m=+0.769112627 container remove d4caab23f6a94101bcb47aad0bfbf7ed15d6673017bb0a433ff429f3b5bd496b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS) 2024-06-03T01:25:51.077 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:50 smithi094 podman[103031]: 2024-06-03 01:25:50.722218342 +0000 UTC m=+0.034993708 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:25:51.077 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:50 smithi094 podman[103031]: 2024-06-03 01:25:50.863670641 +0000 UTC m=+0.176446067 container create e64170be51844f8964838a0e60f9d94ecca4bdd28f91c2972cad905d125424b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-03T01:25:51.349 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:51 smithi094 podman[103031]: 2024-06-03 01:25:51.201592598 +0000 UTC m=+0.514368008 container init e64170be51844f8964838a0e60f9d94ecca4bdd28f91c2972cad905d125424b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True) 2024-06-03T01:25:51.350 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:51 smithi094 podman[103031]: 2024-06-03 01:25:51.21260926 +0000 UTC m=+0.525384673 container start e64170be51844f8964838a0e60f9d94ecca4bdd28f91c2972cad905d125424b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-03T01:25:51.350 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:51 smithi094 podman[103031]: 2024-06-03 01:25:51.308578157 +0000 UTC m=+0.621353600 container attach e64170be51844f8964838a0e60f9d94ecca4bdd28f91c2972cad905d125424b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-03T01:25:52.099 INFO:journalctl@ceph.osd.0.smithi094.stdout:Jun 03 01:25:51 smithi094 podman[103031]: 2024-06-03 01:25:51.664682831 +0000 UTC m=+0.977458224 container died e64170be51844f8964838a0e60f9d94ecca4bdd28f91c2972cad905d125424b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, ceph=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-03T01:25:52.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-03T01:25:52.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:25:52.686 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-03T01:25:52.687 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-03T01:25:52.687 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.1 2024-06-03T01:25:53.017 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:52 smithi107 systemd[1]: Stopping Ceph osd.1 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:25:53.283 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1[41190]: 2024-06-03T01:25:53.016+0000 7f90195b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:25:53.283 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1[41190]: 2024-06-03T01:25:53.016+0000 7f90195b8700 -1 osd.1 123 *** Got signal Terminated *** 2024-06-03T01:25:53.283 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1[41190]: 2024-06-03T01:25:53.016+0000 7f90195b8700 -1 osd.1 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:25:58.694 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:58 smithi107 podman[75067]: 2024-06-03 01:25:58.399943133 +0000 UTC m=+5.510890458 container died d303bf69fedf2812de9ee31687d93ea4f88f53ef74525828c85c8aad9224b8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:25:59.033 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:58 smithi107 podman[75067]: 2024-06-03 01:25:58.792888258 +0000 UTC m=+5.903835582 container cleanup d303bf69fedf2812de9ee31687d93ea4f88f53ef74525828c85c8aad9224b8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-03T01:25:59.033 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:58 smithi107 bash[75067]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1 2024-06-03T01:25:59.374 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:59 smithi107 podman[75084]: 2024-06-03 01:25:59.061431179 +0000 UTC m=+0.655498929 container remove d303bf69fedf2812de9ee31687d93ea4f88f53ef74525828c85c8aad9224b8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-03T01:25:59.648 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:59 smithi107 podman[75175]: 2024-06-03 01:25:59.278444916 +0000 UTC m=+0.016791725 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:25:59.648 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:59 smithi107 podman[75175]: 2024-06-03 01:25:59.396525288 +0000 UTC m=+0.134872124 container create 0d3d64bb1ec6501a028e657843dab64241c0873b047f4a1b5934a8141c078f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-03T01:25:59.648 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:59 smithi107 podman[75175]: 2024-06-03 01:25:59.648799744 +0000 UTC m=+0.387146533 container init 0d3d64bb1ec6501a028e657843dab64241c0873b047f4a1b5934a8141c078f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1-deactivate, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-03T01:25:59.942 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:59 smithi107 podman[75175]: 2024-06-03 01:25:59.652072715 +0000 UTC m=+0.390419512 container start 0d3d64bb1ec6501a028e657843dab64241c0873b047f4a1b5934a8141c078f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-03T01:25:59.943 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:59 smithi107 podman[75175]: 2024-06-03 01:25:59.732257875 +0000 UTC m=+0.470604671 container attach 0d3d64bb1ec6501a028e657843dab64241c0873b047f4a1b5934a8141c078f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-03T01:26:00.251 INFO:journalctl@ceph.osd.1.smithi107.stdout:Jun 03 01:25:59 smithi107 podman[75175]: 2024-06-03 01:25:59.943391983 +0000 UTC m=+0.681738783 container died 0d3d64bb1ec6501a028e657843dab64241c0873b047f4a1b5934a8141c078f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:26:00.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-03T01:26:00.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:26:00.615 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-03T01:26:00.615 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-03T01:26:00.616 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.2 2024-06-03T01:26:00.927 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:00 smithi179 systemd[1]: Stopping Ceph osd.2 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:26:01.188 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2[42154]: 2024-06-03T01:26:00.926+0000 7efc8d7bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:01.189 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2[42154]: 2024-06-03T01:26:00.926+0000 7efc8d7bb700 -1 osd.2 123 *** Got signal Terminated *** 2024-06-03T01:26:01.189 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2[42154]: 2024-06-03T01:26:00.926+0000 7efc8d7bb700 -1 osd.2 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:06.719 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:06 smithi179 podman[75364]: 2024-06-03 01:26:06.39875562 +0000 UTC m=+5.592201843 container died 3e8d9f9aa3a70b998613d679f8e30462252a743a5182b7af2b66b159dd0a4cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-03T01:26:07.103 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:06 smithi179 podman[75364]: 2024-06-03 01:26:06.818782071 +0000 UTC m=+6.012228278 container cleanup 3e8d9f9aa3a70b998613d679f8e30462252a743a5182b7af2b66b159dd0a4cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-03T01:26:07.103 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:06 smithi179 bash[75364]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2 2024-06-03T01:26:07.355 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:07 smithi179 podman[75391]: 2024-06-03 01:26:07.10420638 +0000 UTC m=+0.702837335 container remove 3e8d9f9aa3a70b998613d679f8e30462252a743a5182b7af2b66b159dd0a4cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:26:07.855 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:07 smithi179 podman[75480]: 2024-06-03 01:26:07.490679827 +0000 UTC m=+0.017809201 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:26:07.855 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:07 smithi179 podman[75480]: 2024-06-03 01:26:07.62187874 +0000 UTC m=+0.149008100 container create a9dddad55d5204623d3c3e541de73c19b0e0ddfdcc5e9571fdb146af00b36939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528) 2024-06-03T01:26:08.154 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:07 smithi179 podman[75480]: 2024-06-03 01:26:07.866130017 +0000 UTC m=+0.393259377 container init a9dddad55d5204623d3c3e541de73c19b0e0ddfdcc5e9571fdb146af00b36939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:26:08.154 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:07 smithi179 podman[75480]: 2024-06-03 01:26:07.869226982 +0000 UTC m=+0.396356343 container start a9dddad55d5204623d3c3e541de73c19b0e0ddfdcc5e9571fdb146af00b36939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-03T01:26:08.154 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:07 smithi179 podman[75480]: 2024-06-03 01:26:07.957955707 +0000 UTC m=+0.485085080 container attach a9dddad55d5204623d3c3e541de73c19b0e0ddfdcc5e9571fdb146af00b36939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-03T01:26:08.561 INFO:journalctl@ceph.osd.2.smithi179.stdout:Jun 03 01:26:08 smithi179 podman[75480]: 2024-06-03 01:26:08.155134475 +0000 UTC m=+0.682263841 container died a9dddad55d5204623d3c3e541de73c19b0e0ddfdcc5e9571fdb146af00b36939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-03T01:26:08.880 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-03T01:26:08.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:26:08.921 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-03T01:26:08.921 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-03T01:26:08.922 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.3 2024-06-03T01:26:09.350 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:09 smithi094 systemd[1]: Stopping Ceph osd.3 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:26:09.770 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:09 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T01:26:09.470+0000 7f453fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:09.770 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:09 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T01:26:09.470+0000 7f453fc00700 -1 osd.3 123 *** Got signal Terminated *** 2024-06-03T01:26:09.771 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:09 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T01:26:09.470+0000 7f453fc00700 -1 osd.3 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:09.783 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:09.314+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:09.855 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:09.496+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:10.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:10 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:10.147+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:10.783 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:10.293+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:10.855 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:10.479+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:11.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:10.870+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:11.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:11.280+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:11.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:11 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:11.148+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:11.846 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:11.491+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:12.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:11.847+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:12.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:12 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:12.181+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:12.349 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:12 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T01:26:12.045+0000 7f4536400700 -1 osd.3 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.726891+0000 front 2024-06-03T01:25:47.727661+0000 (oldest deadline 2024-06-03T01:26:11.827220+0000) 2024-06-03T01:26:12.783 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:12.288+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:12.808 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:12.541+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:13.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:12.809+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:13.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:13.178+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:13.349 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:13 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T01:26:13.077+0000 7f4536400700 -1 osd.3 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.726891+0000 front 2024-06-03T01:25:47.727661+0000 (oldest deadline 2024-06-03T01:26:11.827220+0000) 2024-06-03T01:26:13.783 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:13.286+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:13.837 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:13.525+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:14.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:13.837+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:14.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:14 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:14.190+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:14.349 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:14 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3[53837]: 2024-06-03T01:26:14.091+0000 7f4536400700 -1 osd.3 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.726891+0000 front 2024-06-03T01:25:47.727661+0000 (oldest deadline 2024-06-03T01:26:11.827220+0000) 2024-06-03T01:26:14.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:14.246+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:14.827 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:14.517+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:15.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:14.827+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:15.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:15 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:15.201+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:15.349 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:15 smithi094 podman[103136]: 2024-06-03 01:26:15.051234156 +0000 UTC m=+5.722760623 container died d30a9d22116057726193f9373e8c0101f81483f00dbb2e176058e80260c3caee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-03T01:26:15.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:15.214+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:15.811 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:15.476+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:15.849 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:15 smithi094 podman[103136]: 2024-06-03 01:26:15.625343238 +0000 UTC m=+6.296869775 container cleanup d30a9d22116057726193f9373e8c0101f81483f00dbb2e176058e80260c3caee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:26:15.850 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:15 smithi094 bash[103136]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3 2024-06-03T01:26:16.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:15.811+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:16.246 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:16 smithi094 podman[103164]: 2024-06-03 01:26:16.035521483 +0000 UTC m=+0.972872630 container remove d30a9d22116057726193f9373e8c0101f81483f00dbb2e176058e80260c3caee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-03T01:26:16.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:16 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:16.199+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:16.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:16 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:16.245+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:16.849 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:16.456+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:16.916 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:16 smithi094 podman[103255]: 2024-06-03 01:26:16.562075924 +0000 UTC m=+0.033484086 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:26:16.916 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:16 smithi094 podman[103255]: 2024-06-03 01:26:16.736647263 +0000 UTC m=+0.208055464 container create 9c34ac889c4f36901dd8442ffa62884a3f54f1d47b0a1dde909f15d77ba2493e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-03T01:26:17.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:16.849+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:17.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:17 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:17.259+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:17.349 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:17 smithi094 podman[103255]: 2024-06-03 01:26:17.076810154 +0000 UTC m=+0.548218349 container init 9c34ac889c4f36901dd8442ffa62884a3f54f1d47b0a1dde909f15d77ba2493e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-03T01:26:17.350 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:17 smithi094 podman[103255]: 2024-06-03 01:26:17.087557854 +0000 UTC m=+0.558966089 container start 9c34ac889c4f36901dd8442ffa62884a3f54f1d47b0a1dde909f15d77ba2493e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-03T01:26:17.350 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:17 smithi094 podman[103255]: 2024-06-03 01:26:17.181479445 +0000 UTC m=+0.652887716 container attach 9c34ac889c4f36901dd8442ffa62884a3f54f1d47b0a1dde909f15d77ba2493e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:26:17.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:17 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:17.204+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:17.849 INFO:journalctl@ceph.osd.3.smithi094.stdout:Jun 03 01:26:17 smithi094 podman[103281]: 2024-06-03 01:26:17.585909015 +0000 UTC m=+0.033480202 container died 9c34ac889c4f36901dd8442ffa62884a3f54f1d47b0a1dde909f15d77ba2493e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:26:17.855 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:17.426+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:18.220 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:17.862+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:18.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-03T01:26:18.469 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:18 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:18.211+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:18.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:26:18.473 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-03T01:26:18.474 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-03T01:26:18.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.4 2024-06-03T01:26:18.500 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:18.177+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:18.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:18.383+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:18.783 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:18 smithi107 systemd[1]: Stopping Ceph osd.4 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:26:19.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:18.854+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:19.146 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4[47975]: 2024-06-03T01:26:18.799+0000 7fde43e30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:19.146 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4[47975]: 2024-06-03T01:26:18.799+0000 7fde43e30700 -1 osd.4 123 *** Got signal Terminated *** 2024-06-03T01:26:19.146 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:18 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4[47975]: 2024-06-03T01:26:18.799+0000 7fde43e30700 -1 osd.4 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:19.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:19 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:19.147+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:19.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:19 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:19.147+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:19.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:19 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:19.202+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:19.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:19 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:19.202+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:19.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:19.344+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:20.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:19.842+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:20.248 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:19 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:19.982+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:20.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:20 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:20.128+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:20.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:20 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:20.128+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:20.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:20 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:20.247+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:20.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:20 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:20.247+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:20.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:20.332+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:21.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:20.849+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:21.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:20 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:20.965+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:21.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:21.203+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:21.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:21.203+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:21.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:21 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:21.136+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:21.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:21 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:21.136+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:21.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:21.327+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:21.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:21.327+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:22.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:21.851+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:22.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:21.851+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:22.283 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:22 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4[47975]: 2024-06-03T01:26:22.012+0000 7fde3c41c700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:52.944414+0000 front 2024-06-03T01:25:52.944547+0000 (oldest deadline 2024-06-03T01:26:21.745042+0000) 2024-06-03T01:26:22.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:22 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:22.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:22.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:22 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:22.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:22.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:21 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:21.997+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:22.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:22 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:22.225+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:22.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:22 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:22.225+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:22.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:22.357+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:22.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:22.357+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:22.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:22.314+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:23.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:22.844+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:23.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:22.844+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:23.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:23 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:23.015+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:23.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:23 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:23.248+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:23.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:23 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:23.248+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:23.533 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:23 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4[47975]: 2024-06-03T01:26:23.050+0000 7fde3c41c700 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:52.944414+0000 front 2024-06-03T01:25:52.944547+0000 (oldest deadline 2024-06-03T01:26:21.745042+0000) 2024-06-03T01:26:23.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:23 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:23.111+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:23.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:23 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:23.111+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:23.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:23 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:23.274+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:23.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:23 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:23.391+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:23.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:23 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:23.391+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:24.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:23 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:23.854+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:24.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:23 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:23.854+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:24.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:24 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:24.060+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:24.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:24 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:24.221+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:24.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:24 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:24.221+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:24.514 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:24 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:24.068+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:24.515 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:24 smithi107 podman[75293]: 2024-06-03 01:26:24.232040693 +0000 UTC m=+5.563538674 container died fe676de71df37a370d435278f660b2f4c914b1fe016eb2c7ab6471cb08b87f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-03T01:26:24.515 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:24 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:24.102+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:24.515 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:24 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:24.102+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:24.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:24 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:24.266+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:24.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:24 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:24.430+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:24.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:24 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:24.430+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:24.783 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:24 smithi107 podman[75293]: 2024-06-03 01:26:24.613546984 +0000 UTC m=+5.945045010 container cleanup fe676de71df37a370d435278f660b2f4c914b1fe016eb2c7ab6471cb08b87f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-03T01:26:24.783 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:24 smithi107 bash[75293]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4 2024-06-03T01:26:25.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:24 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:24.835+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:25.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:24 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:24.835+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:25.148 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:24 smithi107 podman[75309]: 2024-06-03 01:26:24.88150882 +0000 UTC m=+0.645641456 container remove fe676de71df37a370d435278f660b2f4c914b1fe016eb2c7ab6471cb08b87f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-03T01:26:25.149 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:25 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:25.091+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:25.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:25.085+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:25.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:25.187+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:25.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:25 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:25.187+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:25.476 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:25 smithi107 podman[75400]: 2024-06-03 01:26:25.098634351 +0000 UTC m=+0.017053843 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:26:25.477 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:25 smithi107 podman[75400]: 2024-06-03 01:26:25.199360726 +0000 UTC m=+0.117780192 container create 1a94abfef6776a9f358cc3647bc2d905cbf0c20957a637996acb370c3e4286c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-03T01:26:25.477 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:25 smithi107 podman[75400]: 2024-06-03 01:26:25.477046131 +0000 UTC m=+0.395465616 container init 1a94abfef6776a9f358cc3647bc2d905cbf0c20957a637996acb370c3e4286c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:26:25.477 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:25 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:25.148+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:25.477 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:25 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:25.148+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:25.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:25 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:25.399+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:25.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:25 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:25.399+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:25.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:25 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:25.227+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:25.764 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:25 smithi107 podman[75400]: 2024-06-03 01:26:25.480625736 +0000 UTC m=+0.399045211 container start 1a94abfef6776a9f358cc3647bc2d905cbf0c20957a637996acb370c3e4286c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:26:25.764 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:25 smithi107 podman[75400]: 2024-06-03 01:26:25.577037032 +0000 UTC m=+0.495456513 container attach 1a94abfef6776a9f358cc3647bc2d905cbf0c20957a637996acb370c3e4286c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4-deactivate, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-03T01:26:26.033 INFO:journalctl@ceph.osd.4.smithi107.stdout:Jun 03 01:26:25 smithi107 podman[75400]: 2024-06-03 01:26:25.765369604 +0000 UTC m=+0.683789088 container died 1a94abfef6776a9f358cc3647bc2d905cbf0c20957a637996acb370c3e4286c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-03T01:26:26.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:25 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:25.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:26.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:25 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:25.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:26.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:26.094+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:26.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:26.156+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:26.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:26.156+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:26.414 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:26 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:26.090+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:26.414 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:26 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:26.180+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:26.414 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:26 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:26.180+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:26.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-03T01:26:26.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:26:26.507 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-03T01:26:26.507 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-03T01:26:26.508 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.5 2024-06-03T01:26:26.533 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:26.222+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:26.533 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:26.369+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:26.534 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:26.369+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:26.799 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:26 smithi179 systemd[1]: Stopping Ceph osd.5 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:26:27.105 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:26.831+0000 7f5b2337c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:27.105 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:26.831+0000 7f5b2337c700 -1 osd.5 123 *** Got signal Terminated *** 2024-06-03T01:26:27.105 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:26.831+0000 7f5b2337c700 -1 osd.5 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:27.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:26.799+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:27.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:26.799+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:27.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:26 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:26.799+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:27.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:27.088+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:27.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:27.136+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:27.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:27 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:27.136+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:27.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:27 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:27.099+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:27.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:27 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:27.146+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:27.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:27 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:27.146+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:27.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:27 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:27.391+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:27.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:27 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:27.391+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:27.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:27 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:27.228+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:27.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:27 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:27.228+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:28.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:27 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:27.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:28.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:27 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:27.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:28.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:27 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:27.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:28.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:28 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:28.050+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:28.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:28 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:28.106+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:28.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:28 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:28.106+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:28.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:28 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:28.120+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:28.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:28 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:28.148+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:28.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:28 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:28.148+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:28.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:28.185+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:28.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:28.185+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:28.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:28.417+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:28.606 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:28.417+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:28.606 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:28.417+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.889629+0000 front 2024-06-03T01:26:04.889665+0000 (oldest deadline 2024-06-03T01:26:28.389846+0000) 2024-06-03T01:26:29.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:28.793+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:29.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:28.793+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:29.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:28 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:28.793+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:29.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:29.006+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:29.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:29.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:29.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:29.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:29.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:29 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:29.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:29.436 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:29.137+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:29.436 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:29.137+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:29.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:29 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:29.113+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:29.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:29 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:29.145+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:29.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:29 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:29.145+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:29.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:29 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:29.145+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:29.818 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:29.436+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:29.819 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:29.436+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:29.819 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:29.436+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.889629+0000 front 2024-06-03T01:26:04.889665+0000 (oldest deadline 2024-06-03T01:26:28.389846+0000) 2024-06-03T01:26:30.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:29.817+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:30.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:29.817+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:30.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:29 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:29.818+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:30.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:30.002+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:30.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:30.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:30.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:30.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:30.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:30 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:30.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:30.432 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:30.156+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:30.432 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:30.156+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:30.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:30 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:30.069+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:30.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:30 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:30.069+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:30.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:30 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:30.147+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:30.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:30 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:30.147+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:30.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:30 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:30.147+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:30.776 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:30.431+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:30.776 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:30.431+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:30.776 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:30.431+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.889629+0000 front 2024-06-03T01:26:04.889665+0000 (oldest deadline 2024-06-03T01:26:28.389846+0000) 2024-06-03T01:26:31.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:30.775+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:31.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:30.775+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:31.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:30 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:30.775+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:31.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:31.026+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:31.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:31.038+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:31.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:31.038+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:31.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:31.038+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:31.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:31.195+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:31.447 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:31.195+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:31.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:31 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:31.113+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:31.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:31 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:31.113+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:31.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:31 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:31.132+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:31.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:31 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:31.132+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:31.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:31 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:31.132+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:31.811 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:31.447+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.686606+0000 front 2024-06-03T01:25:45.686625+0000 (oldest deadline 2024-06-03T01:26:09.186507+0000) 2024-06-03T01:26:31.811 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:31.447+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.888660+0000 front 2024-06-03T01:25:57.888825+0000 (oldest deadline 2024-06-03T01:26:20.788775+0000) 2024-06-03T01:26:31.811 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5[48823]: 2024-06-03T01:26:31.447+0000 7f5b1b968700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.889629+0000 front 2024-06-03T01:26:04.889665+0000 (oldest deadline 2024-06-03T01:26:28.389846+0000) 2024-06-03T01:26:32.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:31.810+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:32.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:31.810+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:32.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:31 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:31.810+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:32.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:32.074+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:32.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:32.006+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:32.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:32.006+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:32.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:32 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:32.006+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:32.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:32 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:32.078+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:32.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:32 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:32.078+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:32.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:32 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:32.111+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:32.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:32 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:32.111+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:32.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:32 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:32.111+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:32.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:32 smithi179 podman[75585]: 2024-06-03 01:26:32.422788198 +0000 UTC m=+5.720073894 container died 1b83988e1ece34b0e4427f435a5919f47bef47e4b63fbb237d071c0214b7ebf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-03T01:26:32.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:32 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:32.202+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:32.606 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:32 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:32.202+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:33.058 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:32 smithi179 podman[75585]: 2024-06-03 01:26:32.823742815 +0000 UTC m=+6.121028510 container cleanup 1b83988e1ece34b0e4427f435a5919f47bef47e4b63fbb237d071c0214b7ebf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-03T01:26:33.058 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:32 smithi179 bash[75585]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5 2024-06-03T01:26:33.058 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:32 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:32.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:33.059 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:32 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:32.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:33.059 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:32 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:32.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:33.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:33.044+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:33.350 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:33.044+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:33.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:33.011+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:33.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:33.011+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:33.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:33 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:33.011+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:33.355 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:33 smithi179 podman[75602]: 2024-06-03 01:26:33.058792077 +0000 UTC m=+0.638307073 container remove 1b83988e1ece34b0e4427f435a5919f47bef47e4b63fbb237d071c0214b7ebf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-03T01:26:33.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:33 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:33.215+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:33.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:33 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:33.215+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:33.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:33 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:33.108+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:33.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:33 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:33.108+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:33.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:33 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:33.108+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:33.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:33 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:33.108+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:33.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:33 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:33.108+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:33.711 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:33 smithi179 podman[75689]: 2024-06-03 01:26:33.296270696 +0000 UTC m=+0.017431669 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:26:33.712 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:33 smithi179 podman[75689]: 2024-06-03 01:26:33.401294994 +0000 UTC m=+0.122455949 container create e25f985e65392e0792844953d53d24791e0035e2a68d5b6c57508da511f58a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T01:26:33.712 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:33 smithi179 podman[75689]: 2024-06-03 01:26:33.629171319 +0000 UTC m=+0.350332285 container init e25f985e65392e0792844953d53d24791e0035e2a68d5b6c57508da511f58a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-03T01:26:33.712 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:33 smithi179 podman[75689]: 2024-06-03 01:26:33.632149515 +0000 UTC m=+0.353310471 container start e25f985e65392e0792844953d53d24791e0035e2a68d5b6c57508da511f58a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-03T01:26:34.105 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:33 smithi179 podman[75689]: 2024-06-03 01:26:33.712244647 +0000 UTC m=+0.433405615 container attach e25f985e65392e0792844953d53d24791e0035e2a68d5b6c57508da511f58a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-06-03T01:26:34.106 INFO:journalctl@ceph.osd.5.smithi179.stdout:Jun 03 01:26:33 smithi179 podman[75689]: 2024-06-03 01:26:33.925718347 +0000 UTC m=+0.646879303 container died e25f985e65392e0792844953d53d24791e0035e2a68d5b6c57508da511f58a7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-03T01:26:34.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:33 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:33.839+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:34.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:33 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:33.839+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:34.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:33 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:33.839+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:34.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:34 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:34.075+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:34.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:34 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:34.075+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:34.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:34 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:34.021+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:34.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:34 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:34.021+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:34.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:34 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:34.021+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:34.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:34 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:34.080+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:34.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:34 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:34.080+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:34.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:34 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:34.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:34.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:34 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:34.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:34.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:34 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:34.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:34.590 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:34 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:34.249+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:34.591 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:34 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:34.249+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:34.605 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-03T01:26:34.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:26:34.682 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-03T01:26:34.682 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-03T01:26:34.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.6 2024-06-03T01:26:34.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:34 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:34.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:34.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:34 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:34.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:34.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:34 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:34.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:35.061 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:34 smithi094 systemd[1]: Stopping Ceph osd.6 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:26:35.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:35.061+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:35.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:35.061+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:35.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:35.061+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:35.350 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:35.103+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:35.350 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:35.103+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:35.350 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:35.252+0000 7f1980c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:35.350 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:35.252+0000 7f1980c00700 -1 osd.6 123 *** Got signal Terminated *** 2024-06-03T01:26:35.350 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:35 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:35.252+0000 7f1980c00700 -1 osd.6 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:35.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:35 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:35.115+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:35.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:35 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:35.115+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:35.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:35 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:35.120+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:35.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:35 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:35.120+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:35.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:35 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:35.120+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:35.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:35 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:35.212+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:35.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:35 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:35.212+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:36.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:35 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:35.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:36.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:35 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:35.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:36.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:35 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:35.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:36.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:36.154+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:36.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:36.154+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:36.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:36.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:36.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:36.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:36.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:36.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:36.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:36.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:36.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:36 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:36.128+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:36.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:36 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:36.128+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:36.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:36 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:36.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:36.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:36 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:36.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:36.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:36 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:36.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:36.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:36 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:36.204+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:36.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:36 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:36.204+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:37.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:36 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:36.816+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:37.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:36 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:36.816+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:37.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:36 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:36.816+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:37.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:37 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:37.173+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:37.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:37 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:37.173+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:37.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:37 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:37.173+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:12.248876+0000 front 2024-06-03T01:26:12.248834+0000 (oldest deadline 2024-06-03T01:26:36.848763+0000) 2024-06-03T01:26:37.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:37 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:37.075+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:37.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:37 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:37.075+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:37.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:37 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:37.075+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:37.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:37 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:37.075+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:37.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:37 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:37.109+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:37.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:37 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:37.109+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:37.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:37 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:37.120+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:37.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:37 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:37.120+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:37.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:37 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:37.120+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:37.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:37.185+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:37.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:37.185+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:38.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:37.833+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:38.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:37.833+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:38.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:37 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:37.833+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:38.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:38.129+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:38.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:38.129+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:38.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:38.129+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:12.248876+0000 front 2024-06-03T01:26:12.248834+0000 (oldest deadline 2024-06-03T01:26:36.848763+0000) 2024-06-03T01:26:38.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:38.057+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:38.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:38.057+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:38.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:38.057+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:38.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:38 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:38.057+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:38.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:38 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:38.073+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:38.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:38 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:38.073+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:38.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:38 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:38.073+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:38.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:38 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:38.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:38.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:38 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:38.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:38.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:38 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:38.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:38.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:38 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:38.158+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:38.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:38 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:38.158+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:38.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:38 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:38.158+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:39.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:38 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:38.841+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:39.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:38 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:38.841+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:39.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:38 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:38.841+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:39.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:39.178+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:39.349 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:39.178+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:39.350 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:39.178+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:12.248876+0000 front 2024-06-03T01:26:12.248834+0000 (oldest deadline 2024-06-03T01:26:36.848763+0000) 2024-06-03T01:26:39.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:39.091+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:39.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:39.091+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:39.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:39.091+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:39.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:39.091+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:39.434 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:39 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:39.044+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:39.434 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:39 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:39.044+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:39.435 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:39 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:39.044+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:39.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:39 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:39.175+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:39.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:39 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:39.175+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:39.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:39 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:39.175+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:39.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:39 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:39.201+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:39.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:39 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:39.201+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:39.606 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:39 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:39.201+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:40.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:39 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:39.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:40.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:39 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:39.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:40.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:39 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:39.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:40.285 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:40 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:40.150+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:40.285 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:40 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:40.150+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:40.286 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:40 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:40.150+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:40.286 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:40 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:40.008+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:40.287 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:40 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:40.008+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:40.287 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:40 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:40.008+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:40.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:40.139+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:40.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:40.139+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:40.600 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:40.139+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:40.600 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:40.139+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:40.600 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:40.188+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.042129+0000 front 2024-06-03T01:25:54.042074+0000 (oldest deadline 2024-06-03T01:26:19.942305+0000) 2024-06-03T01:26:40.600 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:40.188+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.245391+0000 front 2024-06-03T01:26:05.245494+0000 (oldest deadline 2024-06-03T01:26:32.247298+0000) 2024-06-03T01:26:40.600 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:40 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6[60368]: 2024-06-03T01:26:40.188+0000 7f1977400700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:12.248876+0000 front 2024-06-03T01:26:12.248834+0000 (oldest deadline 2024-06-03T01:26:36.848763+0000) 2024-06-03T01:26:40.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:40 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:40.156+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:40.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:40 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:40.156+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:40.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:40 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:40.156+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:41.171 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:40 smithi094 podman[103362]: 2024-06-03 01:26:40.862743122 +0000 UTC m=+5.731915494 container stop 8986606ba81ed0bfe6da4c3bd26fbca1a2fccb83e2727da8824961a18b47d481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T01:26:41.172 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:40 smithi094 podman[103362]: 2024-06-03 01:26:40.892101598 +0000 UTC m=+5.761273984 container died 8986606ba81ed0bfe6da4c3bd26fbca1a2fccb83e2727da8824961a18b47d481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-03T01:26:41.184 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:40 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:40.861+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:41.184 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:40 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:40.861+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:41.185 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:40 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:40.861+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:41.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:41 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:41.181+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:41.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:41 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:41.181+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:41.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:41 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:41.181+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:41.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:41 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:41.181+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:41.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:41 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:41.057+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:41.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:41 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:41.057+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:41.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:41 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:41.057+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:41.564 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:41 smithi094 podman[103362]: 2024-06-03 01:26:41.322485036 +0000 UTC m=+6.191657445 container cleanup 8986606ba81ed0bfe6da4c3bd26fbca1a2fccb83e2727da8824961a18b47d481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-03T01:26:41.564 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:41 smithi094 bash[103362]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6 2024-06-03T01:26:41.564 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:41 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:41.171+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:41.565 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:41 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:41.171+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:41.565 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:41 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:41.171+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:41.565 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:41 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:41.171+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:41.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:41 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:41.185+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:41.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:41 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:41.185+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:41.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:41 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:41.185+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:41.849 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:41 smithi094 podman[103378]: 2024-06-03 01:26:41.56231174 +0000 UTC m=+0.681568147 container remove 8986606ba81ed0bfe6da4c3bd26fbca1a2fccb83e2727da8824961a18b47d481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-03T01:26:42.168 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:41 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:41.871+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:42.169 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:41 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:41.871+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:42.169 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:41 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:41.871+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:42.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:42 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:42.225+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:42.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:42 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:42.225+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:42.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:42 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:42.225+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:42.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:42 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:42.225+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:42.284 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:42 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:42.032+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:42.284 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:42 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:42.032+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:42.284 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:42 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:42.032+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:42.425 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:42.136+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:42.425 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:42.136+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:42.425 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:42.136+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:42.425 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:42 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:42.136+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:42.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:42 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:42.169+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:42.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:42 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:42.169+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:42.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:42 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:42.169+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:42.737 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:42 smithi094 podman[103472]: 2024-06-03 01:26:42.329182709 +0000 UTC m=+0.032052931 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:26:42.737 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:42 smithi094 podman[103472]: 2024-06-03 01:26:42.449723515 +0000 UTC m=+0.152593774 container create 1d78748ec8328b9d4d47d9f71b45529bdac5f9c4e719ea217906df6839aaccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-03T01:26:42.737 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:42 smithi094 podman[103472]: 2024-06-03 01:26:42.737443963 +0000 UTC m=+0.440314232 container init 1d78748ec8328b9d4d47d9f71b45529bdac5f9c4e719ea217906df6839aaccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:26:43.099 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:42 smithi094 podman[103472]: 2024-06-03 01:26:42.748598978 +0000 UTC m=+0.451469209 container start 1d78748ec8328b9d4d47d9f71b45529bdac5f9c4e719ea217906df6839aaccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-03T01:26:43.099 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:42 smithi094 podman[103472]: 2024-06-03 01:26:42.819285713 +0000 UTC m=+0.522155965 container attach 1d78748ec8328b9d4d47d9f71b45529bdac5f9c4e719ea217906df6839aaccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-03T01:26:43.193 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:42 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:42.865+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:43.193 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:42 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:42.865+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:43.193 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:42 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:42.865+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:43.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:43.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:43.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:43.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:43.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:43.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:43.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:43.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:43.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:43.074+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:43.535 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:43.074+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:43.535 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:43.074+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:43.545 INFO:journalctl@ceph.osd.6.smithi094.stdout:Jun 03 01:26:43 smithi094 podman[103472]: 2024-06-03 01:26:43.199230164 +0000 UTC m=+0.902100436 container died 1d78748ec8328b9d4d47d9f71b45529bdac5f9c4e719ea217906df6839aaccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-03T01:26:43.545 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:43.116+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:43.545 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:43.116+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:43.545 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:43.116+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:43.546 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:43 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:43.116+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:43.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:43.193+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:43.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:43.193+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:43.606 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:43.193+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:44.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-03T01:26:44.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:26:44.142 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-03T01:26:44.142 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-03T01:26:44.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.7 2024-06-03T01:26:44.155 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:43.879+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:44.155 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:43.879+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:44.155 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:43 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:43.879+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:44.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:44.129+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:44.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:44.129+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:44.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:44.129+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:44.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:44 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:44.129+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:44.491 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:44.071+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:44.491 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:44.071+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:44.491 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:44.071+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:44.491 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:44 smithi107 systemd[1]: Stopping Ceph osd.7 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:26:44.492 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:44.245+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:44.492 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:44.245+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:44.492 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:44.245+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:44.492 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:44.245+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:44.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:44 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:44.155+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:44.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:44 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:44.155+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:44.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:44 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:44.155+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:44.783 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:44.490+0000 7f9cbe5b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:44.783 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:44.490+0000 7f9cbe5b1700 -1 osd.7 123 *** Got signal Terminated *** 2024-06-03T01:26:44.783 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:44 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:44.490+0000 7f9cbe5b1700 -1 osd.7 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:45.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:45 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:45.082+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:45.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:45 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:45.082+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:45.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:45 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:45.082+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:45.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:45 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:45.082+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:45.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:45 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:45.082+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:45.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:44 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:44.895+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:45.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:44 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:44.895+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:45.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:44 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:44.895+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:45.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:45.108+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:45.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:45.108+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:45.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:45.108+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:45.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:45.108+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:45.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:45.082+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:45.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:45.082+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:45.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:45.082+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:45.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:45.203+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:45.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:45.203+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:45.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:45.203+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:45.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:45.203+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:45.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:45 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:45.203+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:46.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:46 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:46.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:46.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:46 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:46.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:46.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:46 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:46.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:46.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:46 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:46.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:46.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:46 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:46.050+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:46.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:46.133+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:46.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:46.133+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:46.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:46.133+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:46.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:46.133+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:46.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:45.925+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:46.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:45.925+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:46.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:45.925+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:46.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:45 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:45.925+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:46.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:46.117+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:46.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:46.117+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:46.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:46.117+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:46.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:46.190+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:46.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:46.190+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:46.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:46.190+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:46.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:46.190+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:46.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:46.190+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:47.147 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:46.890+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:47.148 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:46.890+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:47.148 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:46.890+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:47.148 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:46 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:46.890+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:47.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:47 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:47.046+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:47.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:47 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:47.046+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:47.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:47 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:47.046+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:47.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:47 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:47.046+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:47.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:47 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:47.046+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:47.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:47.093+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:47.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:47.093+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:47.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:47.093+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:47.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:47.214+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:47.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:47.214+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:47.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:47.214+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:47.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:47.214+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:47.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:47 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:47.214+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:47.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:47.148+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:47.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:47.148+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:47.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:47.148+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:47.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:47.148+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:48.184 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:47.872+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:48.185 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:47.872+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:48.185 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:47.872+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:48.185 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:47 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:47.872+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:48.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.045+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:48.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.045+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:48.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.045+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:48.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.045+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:48.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.045+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:48.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:48.046+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:48.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:48.046+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:48.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:48.046+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:48.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:48.046+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:23.638677+0000 front 2024-06-03T01:26:23.638658+0000 (oldest deadline 2024-06-03T01:26:47.138853+0000) 2024-06-03T01:26:48.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:48.258+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:48.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:48.258+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:48.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:48.258+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:48.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:48.258+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:48.535 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:48 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:48.258+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:48.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:48.184+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:48.605 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:48.184+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:48.606 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:48.184+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:48.606 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:48.184+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:49.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.996+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:49.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.996+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:49.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.996+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:49.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.996+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:49.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:48 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:48.996+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:49.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:48.910+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:49.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:48.910+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:49.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:48.910+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:49.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:48 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:48.910+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:49.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:49.153+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:49.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:49.153+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:49.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:49.153+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:49.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:49.153+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:49.533 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:49.069+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:57.334713+0000 front 2024-06-03T01:25:57.334821+0000 (oldest deadline 2024-06-03T01:26:23.234913+0000) 2024-06-03T01:26:49.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:49.069+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:05.536063+0000 front 2024-06-03T01:26:05.536131+0000 (oldest deadline 2024-06-03T01:26:29.636164+0000) 2024-06-03T01:26:49.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:49.069+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.137583+0000 front 2024-06-03T01:26:13.137727+0000 (oldest deadline 2024-06-03T01:26:37.237325+0000) 2024-06-03T01:26:49.534 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7[54783]: 2024-06-03T01:26:49.069+0000 7f9cb6b9d700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:23.638677+0000 front 2024-06-03T01:26:23.638658+0000 (oldest deadline 2024-06-03T01:26:47.138853+0000) 2024-06-03T01:26:49.535 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:49.209+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:49.535 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:49.209+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:49.535 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:49.209+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:49.536 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:49.209+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:49.536 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:49 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:49.209+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:50.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:50.037+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:50.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:50.037+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:50.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:50.037+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:50.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:50.037+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:50.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:50 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:50.037+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:50.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:50.114+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:50.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:50.114+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:50.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:50.114+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:50.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:50.114+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:50.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:49.934+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:50.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:49.934+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:50.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:49.934+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:50.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:49 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:49.934+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:50.366 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:50 smithi107 podman[75518]: 2024-06-03 01:26:50.079578732 +0000 UTC m=+5.737620669 container died f16060dfaa7dae4338a875d33fd9ed61742d99ccf6250e4783d4f9ce8193d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-06-03T01:26:50.366 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:50 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:50.187+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:50.366 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:50 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:50.187+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:50.366 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:50 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:50.187+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:50.367 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:50 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:50.187+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:50.367 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:50 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:50.187+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:50.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:50 smithi107 podman[75518]: 2024-06-03 01:26:50.465829718 +0000 UTC m=+6.123871645 container cleanup f16060dfaa7dae4338a875d33fd9ed61742d99ccf6250e4783d4f9ce8193d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-03T01:26:50.667 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:50 smithi107 bash[75518]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7 2024-06-03T01:26:51.033 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:50 smithi107 podman[75536]: 2024-06-03 01:26:50.667563962 +0000 UTC m=+0.587988025 container remove f16060dfaa7dae4338a875d33fd9ed61742d99ccf6250e4783d4f9ce8193d8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-03T01:26:51.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:51.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:51.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:51.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:51.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:51.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:51.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:51.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:51.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:51 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:51.081+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:51.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:51 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:51.120+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:51.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:51 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:51.120+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:51.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:51 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:51.120+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:51.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:51 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:51.120+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:51.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:50.980+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:51.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:50.980+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:51.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:50.980+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:51.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:50 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:50.980+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:51.379 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:51 smithi107 podman[75627]: 2024-06-03 01:26:51.060284356 +0000 UTC m=+0.092732162 container create f389044b26e8e1c27269a0c40deea34162fc656ac4a4e568b9651d5268484823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-03T01:26:51.379 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:51 smithi107 podman[75627]: 2024-06-03 01:26:50.984275306 +0000 UTC m=+0.016723075 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:26:51.379 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:51 smithi107 podman[75627]: 2024-06-03 01:26:51.27958661 +0000 UTC m=+0.312034372 container init f389044b26e8e1c27269a0c40deea34162fc656ac4a4e568b9651d5268484823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:26:51.379 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:51 smithi107 podman[75627]: 2024-06-03 01:26:51.282604855 +0000 UTC m=+0.315052627 container start f389044b26e8e1c27269a0c40deea34162fc656ac4a4e568b9651d5268484823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-03T01:26:51.379 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:51 smithi107 podman[75627]: 2024-06-03 01:26:51.379533982 +0000 UTC m=+0.411981752 container attach f389044b26e8e1c27269a0c40deea34162fc656ac4a4e568b9651d5268484823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-03T01:26:51.379 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:51 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:51.228+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:51.380 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:51 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:51.228+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:51.380 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:51 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:51.228+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:51.380 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:51 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:51.228+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:51.380 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:51 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:51.228+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:51.783 INFO:journalctl@ceph.osd.7.smithi107.stdout:Jun 03 01:26:51 smithi107 podman[75627]: 2024-06-03 01:26:51.574754081 +0000 UTC m=+0.607201843 container died f389044b26e8e1c27269a0c40deea34162fc656ac4a4e568b9651d5268484823 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-03T01:26:52.206 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-03T01:26:52.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:26:52.244 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-03T01:26:52.244 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-03T01:26:52.244 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.8 2024-06-03T01:26:52.270 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:52.005+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:52.270 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:52.005+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:52.270 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:52.005+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:52.270 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:52.005+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:52.270 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:52.128+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:52.270 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:52.128+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:52.271 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:52.128+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:52.271 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:52.128+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:52.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:52 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:52.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:52.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:52 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:52.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:52.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:52 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:52.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:52.351 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:52 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:52.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:52.351 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:52 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:52.098+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:52.454 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:52 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:52.260+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:52.455 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:52 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:52.260+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:52.455 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:52 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:52.260+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:52.455 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:52 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:52.260+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:52.455 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:52 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:52.260+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:52.568 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 systemd[1]: Stopping Ceph osd.8 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:26:52.855 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:52.568+0000 7fca83a8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:26:52.855 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:52.568+0000 7fca83a8e700 -1 osd.8 123 *** Got signal Terminated *** 2024-06-03T01:26:52.855 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:52 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:52.568+0000 7fca83a8e700 -1 osd.8 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:26:53.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:53.055+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:53.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:53.055+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:53.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:53.055+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:53.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:53.055+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:53.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:53 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:53.055+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:53.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:53.086+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:53.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:53.086+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:53.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:53.086+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:53.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:53.086+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:53.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:53.040+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:53.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:53.040+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:53.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:53.040+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:53.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:53.040+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:53.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:53 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:53.040+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:26:53.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:53.215+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:53.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:53.215+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:53.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:53.215+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:53.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:53.215+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:53.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:53 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:53.215+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:54.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:54.063+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:54.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:54.063+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:54.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:54.063+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:54.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:54.063+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:54.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:54.063+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:54.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:54 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:54.063+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:26:54.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:54.037+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:54.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:54.037+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:54.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:54.037+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:54.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:54.037+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:54.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:54.036+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:54.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:54.036+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:54.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:54.036+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:54.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:54.036+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:54.357 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:54 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:54.036+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:26:54.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:54.172+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:54.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:54.172+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:54.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:54.172+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:54.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:54.172+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:54.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:54.172+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:55.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:55.062+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:55.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:55.062+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:55.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:55.062+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:55.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:55.062+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:55.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:55.062+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:55.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:55 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:55.062+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:26:55.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:55.082+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:55.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:55.082+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:55.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:55.082+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:55.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:55.082+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:55.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:55.032+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:55.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:55.032+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:55.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:55.032+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:55.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:55.032+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:55.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:55 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:55.032+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:26:55.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:55 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:55.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:55.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:55 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:55.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:55.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:55 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:55.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:55.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:55 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:55.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:55.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:55 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:55.162+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:56.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:56.065+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:56.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:56.065+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:56.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:56.065+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:56.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:56.065+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:56.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:56.065+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:56.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:56 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:56.065+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:26:56.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:56.073+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:56.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:56.073+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:56.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:56.073+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:56.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:56.073+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:56.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:56.073+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.137386+0000 front 2024-06-03T01:26:31.137430+0000 (oldest deadline 2024-06-03T01:26:55.237397+0000) 2024-06-03T01:26:56.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:56.046+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:56.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:56.046+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:56.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:56.046+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:56.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:56.046+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:56.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:56 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:56.046+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:26:56.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:56 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:56.168+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:56.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:56 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:56.168+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:56.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:56 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:56.168+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:56.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:56 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:56.168+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:56.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:56 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:56.168+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:57.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:57.086+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:57.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:57.086+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:57.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:57.086+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:57.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:57.086+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:57.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:57.086+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:57.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:57 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:57.086+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:26:57.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:57.047+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.131489+0000 front 2024-06-03T01:25:55.131461+0000 (oldest deadline 2024-06-03T01:26:22.132017+0000) 2024-06-03T01:26:57.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:57.047+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:02.132398+0000 front 2024-06-03T01:26:02.132440+0000 (oldest deadline 2024-06-03T01:26:26.832422+0000) 2024-06-03T01:26:57.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:57.047+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:13.635137+0000 front 2024-06-03T01:26:13.635295+0000 (oldest deadline 2024-06-03T01:26:37.734901+0000) 2024-06-03T01:26:57.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:57.047+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:18.836309+0000 front 2024-06-03T01:26:18.836205+0000 (oldest deadline 2024-06-03T01:26:44.736288+0000) 2024-06-03T01:26:57.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8[55657]: 2024-06-03T01:26:57.047+0000 7fca7c07a700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.137386+0000 front 2024-06-03T01:26:31.137430+0000 (oldest deadline 2024-06-03T01:26:55.237397+0000) 2024-06-03T01:26:57.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:57.008+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:57.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:57.008+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:57.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:57.008+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:57.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:57.008+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:57.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:57 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:57.008+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:26:57.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:57 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:57.216+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:57.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:57 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:57.216+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:57.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:57 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:57.216+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:57.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:57 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:57.216+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:57.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:57 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:57.216+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:58.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:58 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:58.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:58.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:58 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:58.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:58.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:58 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:58.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:58.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:58 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:58.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:58.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:58 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:58.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:58.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:58 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:58.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:26:58.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:58 smithi179 podman[75796]: 2024-06-03 01:26:58.107274989 +0000 UTC m=+5.670232604 container died 16697b8af8b0c691edf9701d0e369bad84222ae6ae61d73470e1c83fb449d3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc) 2024-06-03T01:26:58.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.015+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:58.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.015+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:58.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.015+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:58.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.015+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:58.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.015+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:26:58.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:58 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:58.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:58.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:58 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:58.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:58.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:58 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:58.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:58.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:58 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:58.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:58.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:58 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:58.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:58.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:58 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:58.197+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:26:58.769 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:58 smithi179 podman[75796]: 2024-06-03 01:26:58.501964381 +0000 UTC m=+6.064922037 container cleanup 16697b8af8b0c691edf9701d0e369bad84222ae6ae61d73470e1c83fb449d3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc) 2024-06-03T01:26:58.769 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:58 smithi179 bash[75796]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8 2024-06-03T01:26:59.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:58 smithi179 podman[75811]: 2024-06-03 01:26:58.769919301 +0000 UTC m=+0.659421410 container remove 16697b8af8b0c691edf9701d0e369bad84222ae6ae61d73470e1c83fb449d3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-03T01:26:59.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.972+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:26:59.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.972+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:26:59.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.972+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:26:59.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.972+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:26:59.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:58.972+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:26:59.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:59.043+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:26:59.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:59.043+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:26:59.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:59.043+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:26:59.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:59.043+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:26:59.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:59.043+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:26:59.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:26:59 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:26:59.043+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:26:59.389 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:59 smithi179 podman[75901]: 2024-06-03 01:26:59.02606893 +0000 UTC m=+0.017274308 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:26:59.390 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:59 smithi179 podman[75901]: 2024-06-03 01:26:59.141357332 +0000 UTC m=+0.132562718 container create d4b8f2d083fb60abd11fd7f0988f494f558bc05b5d6baac4279354d99fab8c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-06-03T01:26:59.390 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:59 smithi179 podman[75901]: 2024-06-03 01:26:59.390548191 +0000 UTC m=+0.381753560 container init d4b8f2d083fb60abd11fd7f0988f494f558bc05b5d6baac4279354d99fab8c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True) 2024-06-03T01:26:59.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:59.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:26:59.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:59.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:26:59.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:59.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:26:59.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:59.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:26:59.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:59.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:26:59.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:26:59 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:26:59.153+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:26:59.686 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:59 smithi179 podman[75901]: 2024-06-03 01:26:59.39391058 +0000 UTC m=+0.385115945 container start d4b8f2d083fb60abd11fd7f0988f494f558bc05b5d6baac4279354d99fab8c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:26:59.687 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:59 smithi179 podman[75901]: 2024-06-03 01:26:59.490240786 +0000 UTC m=+0.481446192 container attach d4b8f2d083fb60abd11fd7f0988f494f558bc05b5d6baac4279354d99fab8c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-03T01:27:00.042 INFO:journalctl@ceph.osd.8.smithi179.stdout:Jun 03 01:26:59 smithi179 podman[75901]: 2024-06-03 01:26:59.68735905 +0000 UTC m=+0.678564418 container died d4b8f2d083fb60abd11fd7f0988f494f558bc05b5d6baac4279354d99fab8c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-03T01:27:00.043 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:59.930+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:00.043 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:59.930+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:00.043 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:59.930+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:00.043 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:59.930+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:00.043 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:26:59 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:26:59.930+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:00.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:00.007+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:27:00.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:00.007+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:27:00.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:00.007+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:27:00.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:00.007+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:27:00.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:00.007+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:27:00.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:00 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:00.007+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:27:00.378 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-03T01:27:00.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:00.456 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-03T01:27:00.456 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-03T01:27:00.456 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.9 2024-06-03T01:27:00.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:00.114+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:00.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:00.114+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:00.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:00.114+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:00.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:00.114+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:00.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:00.114+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:00.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:00 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:00.114+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:00.849 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:00 smithi094 systemd[1]: Stopping Ceph osd.9 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:01.227 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:00.882+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:01.227 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:00.882+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:01.227 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:00.882+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:01.227 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:00.882+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:01.227 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:00 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:00.882+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:01.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.015+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:27:01.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.015+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:27:01.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.015+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:27:01.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.015+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:27:01.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.015+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:27:01.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.015+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:27:01.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.074+0000 7fb9ebc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:01.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.074+0000 7fb9ebc00700 -1 osd.9 123 *** Got signal Terminated *** 2024-06-03T01:27:01.351 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:01 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:01.074+0000 7fb9ebc00700 -1 osd.9 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:27:01.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:01.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:01.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:01.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:01.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:01.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:01.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:01.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:01.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:01.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:01.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:01 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:01.113+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:02.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:01.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:02.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:01.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:02.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:01.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:02.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:01.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:02.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:01 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:01.836+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:02.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:02 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:02.056+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:27:02.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:02 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:02.056+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:27:02.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:02 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:02.056+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:27:02.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:02 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:02.056+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:27:02.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:02 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:02.056+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:27:02.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:02 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:02.056+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:27:02.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:02.149+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:02.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:02.149+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:02.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:02.149+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:02.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:02.149+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:02.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:02.149+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:02.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:02.149+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:02.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:02 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:02.149+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:03.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:02.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:03.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:02.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:03.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:02.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:03.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:02.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:03.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:02.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:03.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:02.812+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:03.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:03.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:03.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:03.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:03.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:03.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:03.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:03.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:03.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:03.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:03.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:03.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:03.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:03 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:03.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:03.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:03 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:03.102+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:27:03.601 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:03 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:03.102+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:27:03.601 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:03 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:03.102+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:27:03.601 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:03 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:03.102+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:27:03.601 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:03 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:03.102+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:27:03.601 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:03 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:03.102+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:27:04.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:03.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:04.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:03.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:04.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:03.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:04.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:03.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:04.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:03.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:04.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:03 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:03.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:04.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:04 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:04.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:27:04.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:04 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:04.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:27:04.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:04 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:04.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:27:04.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:04 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:04.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:27:04.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:04 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:04.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:27:04.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:04 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:04.071+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:27:04.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:04.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:04.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:04.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:04.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:04.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:04.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:04.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:04.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:04.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:04.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:04.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:04.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:04 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:04.115+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:05.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:04 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:04.729+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:05.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:04 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:04.729+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:05.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:04 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:04.729+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:05.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:04 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:04.729+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:05.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:04 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:04.729+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:05.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:04 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:04.729+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:05.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:05.067+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:27:05.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:05.067+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:27:05.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:05.067+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:27:05.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:05.067+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:27:05.351 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:05.067+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:27:05.351 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:05 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:05.067+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:27:05.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:05.098+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:05.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:05.098+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:05.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:05.098+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:05.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:05.098+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:05.535 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:05.098+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:05.535 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:05.098+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:05.535 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:05 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:05.098+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:06.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:05 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:05.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:06.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:05 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:05.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:06.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:05 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:05.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:06.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:05 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:05.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:06.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:05 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:05.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:06.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:05 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:05.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:06.349 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:06 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:06.040+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:47.372204+0000 front 2024-06-03T01:25:47.371689+0000 (oldest deadline 2024-06-03T01:26:09.671659+0000) 2024-06-03T01:27:06.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:06 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:06.040+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:56.076064+0000 front 2024-06-03T01:25:56.076251+0000 (oldest deadline 2024-06-03T01:26:18.976269+0000) 2024-06-03T01:27:06.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:06 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:06.040+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:03.678591+0000 front 2024-06-03T01:26:03.678737+0000 (oldest deadline 2024-06-03T01:26:28.978479+0000) 2024-06-03T01:27:06.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:06 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:06.040+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:10.680555+0000 front 2024-06-03T01:26:10.680331+0000 (oldest deadline 2024-06-03T01:26:35.880697+0000) 2024-06-03T01:27:06.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:06 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:06.040+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.883279+0000 front 2024-06-03T01:26:22.883231+0000 (oldest deadline 2024-06-03T01:26:44.583506+0000) 2024-06-03T01:27:06.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:06 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9[67094]: 2024-06-03T01:27:06.040+0000 7fb9e2400700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:29.185895+0000 front 2024-06-03T01:26:29.186051+0000 (oldest deadline 2024-06-03T01:26:53.187712+0000) 2024-06-03T01:27:06.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:06.127+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:06.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:06.127+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:06.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:06.127+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:06.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:06.127+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:06.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:06.127+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:06.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:06.127+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:06.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:06 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:06.127+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:06.941 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:06 smithi094 podman[103577]: 2024-06-03 01:27:06.625453609 +0000 UTC m=+5.747334212 container died 8ba9a60ff7dc3b9a20d02fdebaaa2791793c7334a2e9a8dff41a660da82916bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-03T01:27:07.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:06.780+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:07.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:06.780+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:07.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:06.780+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:07.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:06.780+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:07.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:06.780+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:07.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:06.780+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:07.344 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:07 smithi094 podman[103577]: 2024-06-03 01:27:07.051199161 +0000 UTC m=+6.173079794 container cleanup 8ba9a60ff7dc3b9a20d02fdebaaa2791793c7334a2e9a8dff41a660da82916bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-03T01:27:07.344 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:07 smithi094 bash[103577]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9 2024-06-03T01:27:07.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:07 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:07.082+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:07.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:07 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:07.082+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:07.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:07 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:07.082+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:07.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:07 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:07.082+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:07.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:07 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:07.082+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:07.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:07 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:07.082+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:07.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:07 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:07.082+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:07.599 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:07 smithi094 podman[103596]: 2024-06-03 01:27:07.343744763 +0000 UTC m=+0.702821050 container remove 8ba9a60ff7dc3b9a20d02fdebaaa2791793c7334a2e9a8dff41a660da82916bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-03T01:27:08.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:07 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:07.758+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:08.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:07 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:07.758+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:08.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:07 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:07.758+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:08.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:07 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:07.758+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:08.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:07 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:07.758+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:08.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:07 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:07.758+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:08.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:08 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:08.092+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:08.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:08 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:08.092+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:08.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:08 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:08.092+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:08.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:08 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:08.092+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:08.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:08 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:08.092+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:08.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:08 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:08.092+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:08.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:08 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:08.092+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:08.557 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:08 smithi094 podman[103686]: 2024-06-03 01:27:08.117969144 +0000 UTC m=+0.035598710 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:08.557 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:08 smithi094 podman[103686]: 2024-06-03 01:27:08.290225347 +0000 UTC m=+0.207854934 container create 0959e35bdff8ce51cf934f3b00554ff6c9eb1f1bac63599655f5c70994e188e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-03T01:27:08.557 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:08 smithi094 podman[103686]: 2024-06-03 01:27:08.556853993 +0000 UTC m=+0.474483620 container init 0959e35bdff8ce51cf934f3b00554ff6c9eb1f1bac63599655f5c70994e188e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-03T01:27:08.849 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:08 smithi094 podman[103686]: 2024-06-03 01:27:08.56779711 +0000 UTC m=+0.485426704 container start 0959e35bdff8ce51cf934f3b00554ff6c9eb1f1bac63599655f5c70994e188e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-03T01:27:08.849 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:08 smithi094 podman[103686]: 2024-06-03 01:27:08.71412091 +0000 UTC m=+0.631750551 container attach 0959e35bdff8ce51cf934f3b00554ff6c9eb1f1bac63599655f5c70994e188e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-03T01:27:09.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:08 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:08.747+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:09.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:08 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:08.747+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:09.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:08 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:08.747+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:09.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:08 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:08.747+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:09.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:08 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:08.747+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:09.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:08 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:08.747+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:09.350 INFO:journalctl@ceph.osd.9.smithi094.stdout:Jun 03 01:27:08 smithi094 podman[103686]: 2024-06-03 01:27:08.999422598 +0000 UTC m=+0.917052213 container died 0959e35bdff8ce51cf934f3b00554ff6c9eb1f1bac63599655f5c70994e188e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-06-03T01:27:09.434 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:09.069+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:09.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:09.069+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:09.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:09.069+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:09.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:09.069+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:09.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:09.069+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:09.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:09.069+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:09.436 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:09.069+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:09.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-03T01:27:09.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:09.822 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-03T01:27:09.822 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-03T01:27:09.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.10 2024-06-03T01:27:10.033 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:09 smithi107 systemd[1]: Stopping Ceph osd.10 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:10.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:09.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:10.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:09.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:10.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:09.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:10.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:09.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:10.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:09.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:10.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:09 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:09.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:10.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.056+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:10.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.056+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:10.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.056+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:10.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.056+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:10.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.056+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:10.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.056+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:10.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.056+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:10.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.150+0000 7ff05516e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:10.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.150+0000 7ff05516e700 -1 osd.10 123 *** Got signal Terminated *** 2024-06-03T01:27:10.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:10 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:10.150+0000 7ff05516e700 -1 osd.10 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:27:11.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:10.750+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:11.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:10.750+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:11.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:10.750+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:11.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:10.750+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:11.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:10.750+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:11.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:10.750+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:11.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:11.059+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:11.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:11.059+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:11.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:11.059+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:11.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:11.059+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:11.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:11.059+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:11.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:11.059+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:11.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:11 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:11.059+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:11.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:11.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:11.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:11.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:11.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:11.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:11.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:11.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:11.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:11.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:11.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:11 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:11.757+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:12.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:12.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:12.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:12.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:12.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:12.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:12.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:12.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:12.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:12.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:12.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:12.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:12.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:12 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:12.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:13.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:12.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:13.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:12.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:13.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:12.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:13.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:12.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:13.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:12.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:13.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:12 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:12.774+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:13.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:13.039+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:13.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:13.039+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:13.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:13.039+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:13.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:13.039+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:13.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:13.039+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:13.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:13.039+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:13.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:13 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:13.039+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:14.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:13.815+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:14.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:13.815+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:14.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:13.815+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:14.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:13.815+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:14.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:13.815+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:14.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:13 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:13.815+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:14.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:14.058+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:14.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:14.058+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:14.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:14.058+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:14.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:14.058+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:14.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:14.058+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:14.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:14.058+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:14.534 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:14 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:14.058+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:15.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:14.803+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:15.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:14.803+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:15.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:14.803+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:15.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:14.803+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:15.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:14.803+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:15.107 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:14 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:14.803+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:15.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:15.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:45.082999+0000 front 2024-06-03T01:25:45.083218+0000 (oldest deadline 2024-06-03T01:26:09.182913+0000) 2024-06-03T01:27:15.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:15.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:54.984660+0000 front 2024-06-03T01:25:54.984686+0000 (oldest deadline 2024-06-03T01:26:19.084772+0000) 2024-06-03T01:27:15.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:15.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:04.385872+0000 front 2024-06-03T01:26:04.385953+0000 (oldest deadline 2024-06-03T01:26:29.085726+0000) 2024-06-03T01:27:15.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:15.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-06-03T01:26:14.287485+0000 front 2024-06-03T01:26:14.287419+0000 (oldest deadline 2024-06-03T01:26:40.187114+0000) 2024-06-03T01:27:15.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:15.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:22.488339+0000 front 2024-06-03T01:26:22.488322+0000 (oldest deadline 2024-06-03T01:26:44.788398+0000) 2024-06-03T01:27:15.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:15.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:31.789936+0000 front 2024-06-03T01:26:31.789987+0000 (oldest deadline 2024-06-03T01:26:57.690049+0000) 2024-06-03T01:27:15.284 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10[61650]: 2024-06-03T01:27:15.025+0000 7ff04d75a700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:37.690775+0000 front 2024-06-03T01:26:37.690875+0000 (oldest deadline 2024-06-03T01:27:01.190616+0000) 2024-06-03T01:27:15.975 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 podman[75752]: 2024-06-03 01:27:15.662718031 +0000 UTC m=+5.640851128 container died 653c01c54942b450a504212625f38c7812575f4d3812ea0f1bf0448898cb0598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-03T01:27:15.975 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 podman[75752]: 2024-06-03 01:27:15.975700735 +0000 UTC m=+5.953833841 container cleanup 653c01c54942b450a504212625f38c7812575f4d3812ea0f1bf0448898cb0598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:16.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:15.830+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:16.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:15.830+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:16.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:15.830+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:16.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:15.830+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:16.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:15.830+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:16.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:15.830+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:16.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:15 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:15.830+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:16.251 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:15 smithi107 bash[75752]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10 2024-06-03T01:27:16.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:16 smithi107 podman[75770]: 2024-06-03 01:27:16.252212135 +0000 UTC m=+0.585379902 container remove 653c01c54942b450a504212625f38c7812575f4d3812ea0f1bf0448898cb0598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-03T01:27:16.946 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:16 smithi107 podman[75860]: 2024-06-03 01:27:16.6449808 +0000 UTC m=+0.108944508 container create 308fb120ef0ed3606159b4be18f8b7235bcaabc5f4b741af22d86a20fdfae255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-03T01:27:16.947 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:16 smithi107 podman[75860]: 2024-06-03 01:27:16.553369441 +0000 UTC m=+0.017333170 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:16.947 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:16 smithi107 podman[75860]: 2024-06-03 01:27:16.880695869 +0000 UTC m=+0.344659581 container init 308fb120ef0ed3606159b4be18f8b7235bcaabc5f4b741af22d86a20fdfae255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-03T01:27:16.947 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:16 smithi107 podman[75860]: 2024-06-03 01:27:16.883721058 +0000 UTC m=+0.347684767 container start 308fb120ef0ed3606159b4be18f8b7235bcaabc5f4b741af22d86a20fdfae255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:16.947 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:16 smithi107 podman[75860]: 2024-06-03 01:27:16.947134917 +0000 UTC m=+0.411098640 container attach 308fb120ef0ed3606159b4be18f8b7235bcaabc5f4b741af22d86a20fdfae255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:17.283 INFO:journalctl@ceph.osd.10.smithi107.stdout:Jun 03 01:27:17 smithi107 podman[75860]: 2024-06-03 01:27:17.174681404 +0000 UTC m=+0.638645120 container died 308fb120ef0ed3606159b4be18f8b7235bcaabc5f4b741af22d86a20fdfae255 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-03T01:27:17.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:16.855+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:17.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:16.855+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:17.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:16.855+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:17.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:16.855+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:17.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:16.855+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:17.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:16.855+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:17.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:16 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:16.855+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:17.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-03T01:27:17.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:17.781 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-03T01:27:17.781 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-03T01:27:17.781 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.11 2024-06-03T01:27:18.088 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:17.821+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:18.088 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:17.821+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:18.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:17.821+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:18.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:17.821+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:18.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:17.821+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:18.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:17.821+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:18.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:17.821+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:18.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:17 smithi179 systemd[1]: Stopping Ceph osd.11 for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:18.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.087+0000 7fae96fbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:18.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.087+0000 7fae96fbf700 -1 osd.11 123 *** Got signal Terminated *** 2024-06-03T01:27:18.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.087+0000 7fae96fbf700 -1 osd.11 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-03T01:27:19.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:19.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:19.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:19.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:19.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:19.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:19.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:18 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:18.824+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:20.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:20.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:20.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:20.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:20.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:20.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:20.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:20.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:19 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:19.825+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-06-03T01:26:55.530808+0000 front 2024-06-03T01:26:55.530839+0000 (oldest deadline 2024-06-03T01:27:19.630837+0000) 2024-06-03T01:27:21.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:21.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:21.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:21.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:21.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:21.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:21.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:21.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:20 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:20.828+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-06-03T01:26:55.530808+0000 front 2024-06-03T01:26:55.530839+0000 (oldest deadline 2024-06-03T01:27:19.630837+0000) 2024-06-03T01:27:22.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:22.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:22.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:22.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:22.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:22.357 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:22.357 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:22.357 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:21 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:21.868+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-06-03T01:26:55.530808+0000 front 2024-06-03T01:26:55.530839+0000 (oldest deadline 2024-06-03T01:27:19.630837+0000) 2024-06-03T01:27:23.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-06-03T01:25:44.018895+0000 front 2024-06-03T01:25:44.019120+0000 (oldest deadline 2024-06-03T01:26:09.918812+0000) 2024-06-03T01:27:23.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-06-03T01:25:55.621163+0000 front 2024-06-03T01:25:55.621144+0000 (oldest deadline 2024-06-03T01:26:21.521128+0000) 2024-06-03T01:27:23.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-06-03T01:26:01.521455+0000 front 2024-06-03T01:26:01.521575+0000 (oldest deadline 2024-06-03T01:26:26.722351+0000) 2024-06-03T01:27:23.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-06-03T01:26:19.425752+0000 front 2024-06-03T01:26:19.425650+0000 (oldest deadline 2024-06-03T01:26:45.325676+0000) 2024-06-03T01:27:23.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-06-03T01:26:30.427496+0000 front 2024-06-03T01:26:30.427587+0000 (oldest deadline 2024-06-03T01:26:52.727624+0000) 2024-06-03T01:27:23.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-06-03T01:26:39.129295+0000 front 2024-06-03T01:26:39.129359+0000 (oldest deadline 2024-06-03T01:27:02.028992+0000) 2024-06-03T01:27:23.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-06-03T01:26:46.730018+0000 front 2024-06-03T01:26:46.730063+0000 (oldest deadline 2024-06-03T01:27:15.530480+0000) 2024-06-03T01:27:23.356 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:22 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11[62221]: 2024-06-03T01:27:22.889+0000 7fae8f5ab700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-06-03T01:26:55.530808+0000 front 2024-06-03T01:26:55.530839+0000 (oldest deadline 2024-06-03T01:27:19.630837+0000) 2024-06-03T01:27:23.714 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:23 smithi179 podman[76020]: 2024-06-03 01:27:23.443963239 +0000 UTC m=+5.468264597 container died 468a060ffddc4b111fddda9b7bf8e798d8f6098c72deb6f8ecf0927b4d29bdaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-03T01:27:24.055 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:23 smithi179 podman[76020]: 2024-06-03 01:27:23.838076398 +0000 UTC m=+5.862377762 container cleanup 468a060ffddc4b111fddda9b7bf8e798d8f6098c72deb6f8ecf0927b4d29bdaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-06-03T01:27:24.055 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:23 smithi179 bash[76020]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11 2024-06-03T01:27:24.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:24 smithi179 podman[76036]: 2024-06-03 01:27:24.05593664 +0000 UTC m=+0.614063954 container remove 468a060ffddc4b111fddda9b7bf8e798d8f6098c72deb6f8ecf0927b4d29bdaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-03T01:27:24.675 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:24 smithi179 podman[76126]: 2024-06-03 01:27:24.285162082 +0000 UTC m=+0.017404073 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-03T01:27:24.675 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:24 smithi179 podman[76126]: 2024-06-03 01:27:24.389963113 +0000 UTC m=+0.122205084 container create 88a824f4137f7350587784ac8bff5053cc258814cfe943a6a31510a686eaf362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:27:24.675 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:24 smithi179 podman[76126]: 2024-06-03 01:27:24.592030668 +0000 UTC m=+0.324272640 container init 88a824f4137f7350587784ac8bff5053cc258814cfe943a6a31510a686eaf362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-03T01:27:24.675 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:24 smithi179 podman[76126]: 2024-06-03 01:27:24.595367348 +0000 UTC m=+0.327609322 container start 88a824f4137f7350587784ac8bff5053cc258814cfe943a6a31510a686eaf362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:24.675 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:24 smithi179 podman[76126]: 2024-06-03 01:27:24.675437375 +0000 UTC m=+0.407679362 container attach 88a824f4137f7350587784ac8bff5053cc258814cfe943a6a31510a686eaf362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-03T01:27:25.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:24 smithi179 podman[76126]: 2024-06-03 01:27:24.889012426 +0000 UTC m=+0.621254411 container died 88a824f4137f7350587784ac8bff5053cc258814cfe943a6a31510a686eaf362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-03T01:27:25.459 INFO:journalctl@ceph.osd.11.smithi179.stdout:Jun 03 01:27:25 smithi179 podman[76153]: 2024-06-03 01:27:25.445359919 +0000 UTC m=+0.550740076 container remove 88a824f4137f7350587784ac8bff5053cc258814cfe943a6a31510a686eaf362 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-osd-11-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-03T01:27:25.464 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-03T01:27:25.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:25.502 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-03T01:27:25.502 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-03T01:27:25.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.a 2024-06-03T01:27:25.849 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:27:25 smithi094 systemd[1]: Stopping Ceph mds.a for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:26.349 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:27:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a[74154]: 2024-06-03T01:27:26.060+0000 7f460ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:26.349 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:27:26 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a[74154]: 2024-06-03T01:27:26.060+0000 7f460ea00700 -1 mds.a *** got signal Terminated *** 2024-06-03T01:27:29.849 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:27:29 smithi094 podman[103802]: 2024-06-03 01:27:29.543681424 +0000 UTC m=+3.617271583 container died 9f54f277a773ee40ee34e26ab5aac76a06d5491abae76c9f220512613dd9b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-03T01:27:30.238 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:27:29 smithi094 podman[103802]: 2024-06-03 01:27:29.98653668 +0000 UTC m=+4.060126833 container cleanup 9f54f277a773ee40ee34e26ab5aac76a06d5491abae76c9f220512613dd9b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-06-03T01:27:30.238 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:27:29 smithi094 bash[103802]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a 2024-06-03T01:27:30.599 INFO:journalctl@ceph.mds.a.smithi094.stdout:Jun 03 01:27:30 smithi094 podman[103816]: 2024-06-03 01:27:30.237768055 +0000 UTC m=+0.679014764 container remove 9f54f277a773ee40ee34e26ab5aac76a06d5491abae76c9f220512613dd9b2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True) 2024-06-03T01:27:30.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-03T01:27:30.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:30.811 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-03T01:27:30.811 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-03T01:27:30.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.d 2024-06-03T01:27:31.349 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:27:30 smithi094 systemd[1]: Stopping Ceph mds.d for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:31.849 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:27:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d[72973]: 2024-06-03T01:27:31.388+0000 7f1c1bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:31.849 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:27:31 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d[72973]: 2024-06-03T01:27:31.388+0000 7f1c1bc00700 -1 mds.d *** got signal Terminated *** 2024-06-03T01:27:34.822 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:27:34 smithi094 podman[103923]: 2024-06-03 01:27:34.521846547 +0000 UTC m=+3.256525097 container died 95656a1bf6c1de201bc352574d965f7671b2e735712ec1ebcd661aa1da6317d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-03T01:27:35.099 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:27:34 smithi094 podman[103923]: 2024-06-03 01:27:34.932628869 +0000 UTC m=+3.667307373 container cleanup 95656a1bf6c1de201bc352574d965f7671b2e735712ec1ebcd661aa1da6317d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-03T01:27:35.099 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:27:34 smithi094 bash[103923]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d 2024-06-03T01:27:35.599 INFO:journalctl@ceph.mds.d.smithi094.stdout:Jun 03 01:27:35 smithi094 podman[103938]: 2024-06-03 01:27:35.208721989 +0000 UTC m=+0.671982289 container remove 95656a1bf6c1de201bc352574d965f7671b2e735712ec1ebcd661aa1da6317d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-03T01:27:35.754 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-03T01:27:35.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:35.855 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-03T01:27:35.856 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-03T01:27:35.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.g 2024-06-03T01:27:36.350 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:27:36 smithi094 systemd[1]: Stopping Ceph mds.g for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:36.849 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:27:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g[73749]: 2024-06-03T01:27:36.459+0000 7f18f0e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:36.849 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:27:36 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g[73749]: 2024-06-03T01:27:36.459+0000 7f18f0e00700 -1 mds.g *** got signal Terminated *** 2024-06-03T01:27:37.849 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:27:37 smithi094 podman[104042]: 2024-06-03 01:27:37.552639747 +0000 UTC m=+1.218871134 container died 3daa0df28bb214e4880d4c8d1d5cbd15f7322362c1acc34511b4e8c688ef38b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-03T01:27:38.263 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:27:37 smithi094 podman[104042]: 2024-06-03 01:27:37.986386701 +0000 UTC m=+1.652618081 container cleanup 3daa0df28bb214e4880d4c8d1d5cbd15f7322362c1acc34511b4e8c688ef38b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-03T01:27:38.263 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:27:37 smithi094 bash[104042]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g 2024-06-03T01:27:38.599 INFO:journalctl@ceph.mds.g.smithi094.stdout:Jun 03 01:27:38 smithi094 podman[104053]: 2024-06-03 01:27:38.26281601 +0000 UTC m=+0.697988466 container remove 3daa0df28bb214e4880d4c8d1d5cbd15f7322362c1acc34511b4e8c688ef38b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:38.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-03T01:27:38.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:38.889 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-03T01:27:38.889 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-03T01:27:38.890 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.j 2024-06-03T01:27:39.348 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:27:39 smithi094 systemd[1]: Stopping Ceph mds.j for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:39.715 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:27:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j[72559]: 2024-06-03T01:27:39.521+0000 7f191d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:39.715 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:27:39 smithi094 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j[72559]: 2024-06-03T01:27:39.521+0000 7f191d000700 -1 mds.j *** got signal Terminated *** 2024-06-03T01:27:42.114 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:27:41 smithi094 podman[104159]: 2024-06-03 01:27:41.850525902 +0000 UTC m=+2.524525623 container died 85d667e7ee530d0aacc6c276c8727b189462c19871b9eb6731c65e4811cbe7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:27:42.467 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:27:42 smithi094 podman[104159]: 2024-06-03 01:27:42.307654569 +0000 UTC m=+2.981654283 container cleanup 85d667e7ee530d0aacc6c276c8727b189462c19871b9eb6731c65e4811cbe7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-03T01:27:42.467 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:27:42 smithi094 bash[104159]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j 2024-06-03T01:27:42.842 INFO:journalctl@ceph.mds.j.smithi094.stdout:Jun 03 01:27:42 smithi094 podman[104181]: 2024-06-03 01:27:42.467022509 +0000 UTC m=+0.594761652 container remove 85d667e7ee530d0aacc6c276c8727b189462c19871b9eb6731c65e4811cbe7df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-03T01:27:42.877 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-03T01:27:42.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:42.985 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-03T01:27:42.986 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-03T01:27:42.986 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.b 2024-06-03T01:27:43.283 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:27:43 smithi107 systemd[1]: Stopping Ceph mds.b for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:43.541 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:27:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: 2024-06-03T01:27:43.289+0000 7f11001a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:43.541 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:27:43 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b[67010]: 2024-06-03T01:27:43.289+0000 7f11001a8700 -1 mds.b *** got signal Terminated *** 2024-06-03T01:27:45.783 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:27:45 smithi107 podman[75964]: 2024-06-03 01:27:45.513203317 +0000 UTC m=+2.328072269 container died ab857863a9e0ae32331fe86e58ab955f464b11ef4d5b8984cd03e16f1fdb171c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:27:46.149 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:27:45 smithi107 podman[75964]: 2024-06-03 01:27:45.92414663 +0000 UTC m=+2.739015583 container cleanup ab857863a9e0ae32331fe86e58ab955f464b11ef4d5b8984cd03e16f1fdb171c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-03T01:27:46.150 INFO:journalctl@ceph.mds.b.smithi107.stdout:Jun 03 01:27:45 smithi107 bash[75964]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-b 2024-06-03T01:27:46.281 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-03T01:27:46.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:46.315 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-03T01:27:46.315 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-03T01:27:46.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.e 2024-06-03T01:27:46.533 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:27:46 smithi107 systemd[1]: Stopping Ceph mds.e for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:47.033 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:27:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e[66397]: 2024-06-03T01:27:46.627+0000 7fbd3aab2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:47.033 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:27:46 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e[66397]: 2024-06-03T01:27:46.627+0000 7fbd3aab2700 -1 mds.e *** got signal Terminated *** 2024-06-03T01:27:49.783 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:27:49 smithi107 podman[76096]: 2024-06-03 01:27:49.508991126 +0000 UTC m=+2.978118974 container died b23cdaba92a445658ea3e910019f9fc89af5e94a6ec94507cad41aa6c5f3e99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=) 2024-06-03T01:27:50.283 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:27:50 smithi107 podman[76096]: 2024-06-03 01:27:50.078489003 +0000 UTC m=+3.547616840 container cleanup b23cdaba92a445658ea3e910019f9fc89af5e94a6ec94507cad41aa6c5f3e99a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-03T01:27:50.283 INFO:journalctl@ceph.mds.e.smithi107.stdout:Jun 03 01:27:50 smithi107 bash[76096]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-e 2024-06-03T01:27:50.485 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-03T01:27:50.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:50.517 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-03T01:27:50.517 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-03T01:27:50.517 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.h 2024-06-03T01:27:50.865 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:50 smithi107 systemd[1]: Stopping Ceph mds.h for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:51.283 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:50 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h[66699]: 2024-06-03T01:27:50.865+0000 7fde759de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:51.283 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:50 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h[66699]: 2024-06-03T01:27:50.865+0000 7fde759de700 -1 mds.h *** got signal Terminated *** 2024-06-03T01:27:53.783 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:53 smithi107 podman[76217]: 2024-06-03 01:27:53.507851411 +0000 UTC m=+2.765641161 container died a5069b67326673ece68b0cdf30f815cb1f2248140166ad7befd4bc9c39cdcad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-03T01:27:54.124 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:53 smithi107 podman[76217]: 2024-06-03 01:27:53.923837601 +0000 UTC m=+3.181627354 container cleanup a5069b67326673ece68b0cdf30f815cb1f2248140166ad7befd4bc9c39cdcad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-03T01:27:54.125 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:53 smithi107 bash[76217]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h 2024-06-03T01:27:54.357 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-03T01:27:54.377 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:54 smithi107 podman[76232]: 2024-06-03 01:27:54.125259434 +0000 UTC m=+0.619566719 container remove a5069b67326673ece68b0cdf30f815cb1f2248140166ad7befd4bc9c39cdcad1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-03T01:27:54.377 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:54 smithi107 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.h.service: Deactivated successfully. 2024-06-03T01:27:54.377 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:54 smithi107 systemd[1]: Stopped Ceph mds.h for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:27:54.378 INFO:journalctl@ceph.mds.h.smithi107.stdout:Jun 03 01:27:54 smithi107 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.h.service: Consumed 47.295s CPU time. 2024-06-03T01:27:54.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:54.388 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-03T01:27:54.388 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-03T01:27:54.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.k 2024-06-03T01:27:54.710 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:27:54 smithi107 systemd[1]: Stopping Ceph mds.k for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:55.033 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:27:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k[66083]: 2024-06-03T01:27:54.710+0000 7fbbdc997700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:55.033 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:27:54 smithi107 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k[66083]: 2024-06-03T01:27:54.710+0000 7fbbdc997700 -1 mds.k *** got signal Terminated *** 2024-06-03T01:27:57.783 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:27:57 smithi107 podman[76339]: 2024-06-03 01:27:57.497455925 +0000 UTC m=+2.890541580 container died fb8529a3db0be1c3ae825ecf39066b007adbe4444b519205e9329b569508bc1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-03T01:27:58.111 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:27:57 smithi107 podman[76339]: 2024-06-03 01:27:57.86087722 +0000 UTC m=+3.253962875 container cleanup fb8529a3db0be1c3ae825ecf39066b007adbe4444b519205e9329b569508bc1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-03T01:27:58.111 INFO:journalctl@ceph.mds.k.smithi107.stdout:Jun 03 01:27:57 smithi107 bash[76339]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-k 2024-06-03T01:27:58.228 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-03T01:27:58.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:27:58.259 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-03T01:27:58.259 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-03T01:27:58.259 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.c 2024-06-03T01:27:58.570 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:27:58 smithi179 systemd[1]: Stopping Ceph mds.c for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:27:58.855 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:27:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c[67635]: 2024-06-03T01:27:58.594+0000 7fb1f3a72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:27:58.855 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:27:58 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c[67635]: 2024-06-03T01:27:58.594+0000 7fb1f3a72700 -1 mds.c *** got signal Terminated *** 2024-06-03T01:28:01.779 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:28:01 smithi179 podman[76232]: 2024-06-03 01:28:01.499199581 +0000 UTC m=+3.043979288 container died 18ef3204d40437389fa34fa41e264f0ca277a2d00d71dce6e294f29db3ba97f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-03T01:28:02.105 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:28:01 smithi179 podman[76232]: 2024-06-03 01:28:01.861752631 +0000 UTC m=+3.406532321 container cleanup 18ef3204d40437389fa34fa41e264f0ca277a2d00d71dce6e294f29db3ba97f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-03T01:28:02.105 INFO:journalctl@ceph.mds.c.smithi179.stdout:Jun 03 01:28:01 smithi179 bash[76232]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-c 2024-06-03T01:28:02.302 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-03T01:28:02.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:28:02.337 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-03T01:28:02.337 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-03T01:28:02.338 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.f 2024-06-03T01:28:02.855 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:02 smithi179 systemd[1]: Stopping Ceph mds.f for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:28:02.855 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f[67330]: 2024-06-03T01:28:02.656+0000 7f652b354700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:28:02.856 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:02 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f[67330]: 2024-06-03T01:28:02.656+0000 7f652b354700 -1 mds.f *** got signal Terminated *** 2024-06-03T01:28:05.766 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:05 smithi179 podman[76366]: 2024-06-03 01:28:05.495254461 +0000 UTC m=+2.940753752 container died 11b6c45346f147de94057d488e52d3b97415d1acac0e2c2d856040591fa8a41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-03T01:28:06.033 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:05 smithi179 podman[76366]: 2024-06-03 01:28:05.848986736 +0000 UTC m=+3.294486021 container cleanup 11b6c45346f147de94057d488e52d3b97415d1acac0e2c2d856040591fa8a41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc) 2024-06-03T01:28:06.033 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:05 smithi179 bash[76366]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f 2024-06-03T01:28:06.261 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-03T01:28:06.283 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:06 smithi179 podman[76378]: 2024-06-03 01:28:06.03350151 +0000 UTC m=+0.527932547 container remove 11b6c45346f147de94057d488e52d3b97415d1acac0e2c2d856040591fa8a41e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-03T01:28:06.283 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:06 smithi179 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.f.service: Deactivated successfully. 2024-06-03T01:28:06.284 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:06 smithi179 systemd[1]: Stopped Ceph mds.f for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:28:06.284 INFO:journalctl@ceph.mds.f.smithi179.stdout:Jun 03 01:28:06 smithi179 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.f.service: Consumed 24.880s CPU time. 2024-06-03T01:28:06.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:28:06.294 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-03T01:28:06.294 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-03T01:28:06.294 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.i 2024-06-03T01:28:06.605 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:28:06 smithi179 systemd[1]: Stopping Ceph mds.i for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:28:07.105 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:28:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i[67952]: 2024-06-03T01:28:06.614+0000 7f3fd03ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:28:07.105 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:28:06 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i[67952]: 2024-06-03T01:28:06.614+0000 7f3fd03ee700 -1 mds.i *** got signal Terminated *** 2024-06-03T01:28:09.855 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:28:09 smithi179 podman[76490]: 2024-06-03 01:28:09.495108864 +0000 UTC m=+2.980360880 container died e0f7f4e23e85c9fb5c11e98394b64865c6fb94962261b8c6b5252f649691e417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-03T01:28:10.172 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:28:09 smithi179 podman[76490]: 2024-06-03 01:28:09.947166674 +0000 UTC m=+3.432418689 container cleanup e0f7f4e23e85c9fb5c11e98394b64865c6fb94962261b8c6b5252f649691e417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-03T01:28:10.172 INFO:journalctl@ceph.mds.i.smithi179.stdout:Jun 03 01:28:09 smithi179 bash[76490]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-i 2024-06-03T01:28:10.295 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-03T01:28:10.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:28:10.326 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-03T01:28:10.326 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-03T01:28:10.327 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.l 2024-06-03T01:28:10.605 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:10 smithi179 systemd[1]: Stopping Ceph mds.l for d037ccd0-2143-11ef-bc9b-c7b262605968... 2024-06-03T01:28:11.105 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l[67028]: 2024-06-03T01:28:10.642+0000 7f797703f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-03T01:28:11.105 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:10 smithi179 ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l[67028]: 2024-06-03T01:28:10.642+0000 7f797703f700 -1 mds.l *** got signal Terminated *** 2024-06-03T01:28:13.855 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:13 smithi179 podman[76611]: 2024-06-03 01:28:13.501195025 +0000 UTC m=+2.958979334 container died c7fcb93dc6251452a82d47af08242fdbc24d52dcb7fcb7aaefab61510730b63a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-03T01:28:14.126 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:13 smithi179 podman[76611]: 2024-06-03 01:28:13.94252309 +0000 UTC m=+3.400307391 container cleanup c7fcb93dc6251452a82d47af08242fdbc24d52dcb7fcb7aaefab61510730b63a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-03T01:28:14.126 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:13 smithi179 bash[76611]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l 2024-06-03T01:28:14.368 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-03T01:28:14.387 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:14 smithi179 podman[76625]: 2024-06-03 01:28:14.126746421 +0000 UTC m=+0.625532014 container remove c7fcb93dc6251452a82d47af08242fdbc24d52dcb7fcb7aaefab61510730b63a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-d037ccd0-2143-11ef-bc9b-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-03T01:28:14.388 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:14 smithi179 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.l.service: Deactivated successfully. 2024-06-03T01:28:14.388 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:14 smithi179 systemd[1]: Stopped Ceph mds.l for d037ccd0-2143-11ef-bc9b-c7b262605968. 2024-06-03T01:28:14.388 INFO:journalctl@ceph.mds.l.smithi179.stdout:Jun 03 01:28:14 smithi179 systemd[1]: ceph-d037ccd0-2143-11ef-bc9b-c7b262605968@mds.l.service: Consumed 5.506s CPU time. 2024-06-03T01:28:14.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-03T01:28:14.398 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-03T01:28:14.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T01:28:14.720 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T01:28:19.614 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T01:28:19.801 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T01:28:22.254 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-03T01:28:22.442 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T01:28:24.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:28:24.819 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:28:24.851 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-03T01:28:24.876 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-03T01:28:24.879 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi094/crash 2024-06-03T01:28:24.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T01:28:24.981 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T01:28:24.981 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-06-03T01:28:24.985 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi107/crash 2024-06-03T01:28:24.986 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T01:28:25.108 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T01:28:25.108 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-06-03T01:28:25.110 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi179/crash 2024-06-03T01:28:25.111 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash -- . 2024-06-03T01:28:25.146 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-03T01:28:25.147 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-06-03T01:28:25.148 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-03T01:28:25.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-06-03T01:28:25.251 INFO:tasks.cephadm:Compressing logs... 2024-06-03T01:28:25.251 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T01:28:25.255 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T01:28:25.258 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-03T01:28:25.282 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T01:28:25.282 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-06-03T01:28:25.282 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-03T01:28:25.282 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-03T01:28:25.282 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T01:28:25.283 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-03T01:28:25.283 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-03T01:28:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-03T01:28:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-03T01:28:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-03T01:28:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.z.log: 87.1% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T01:28:25.291 INFO:teuthology.orchestra.run.smithi179.stderr: 90.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T01:28:25.291 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T01:28:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-03T01:28:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T01:28:25.292 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-03T01:28:25.292 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.b.log/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log: 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-03T01:28:25.293 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-03T01:28:25.294 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-03T01:28:25.294 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T01:28:25.294 INFO:teuthology.orchestra.run.smithi107.stderr:gzip 87.1% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T01:28:25.294 INFO:teuthology.orchestra.run.smithi107.stderr: -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T01:28:25.294 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-03T01:28:25.294 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T01:28:25.295 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.11.log: 92.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-03T01:28:25.296 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T01:28:25.296 INFO:teuthology.orchestra.run.smithi107.stderr: 92.3% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-03T01:28:25.298 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T01:28:25.298 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-03T01:28:25.306 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-03T01:28:25.312 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T01:28:25.314 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-03T01:28:25.314 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log 2024-06-03T01:28:25.315 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-03T01:28:25.316 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-03T01:28:25.317 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-03T01:28:25.318 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-03T01:28:25.319 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.j.log: 87.1% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-03T01:28:25.319 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-03T01:28:25.321 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-03T01:28:25.321 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.3.log: 90.2% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-03T01:28:25.322 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-03T01:28:25.324 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-03T01:28:25.324 INFO:teuthology.orchestra.run.smithi094.stderr: 94.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-03T01:28:25.325 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-03T01:28:25.325 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-03T01:28:25.327 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-03T01:28:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T01:28:25.328 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-03T01:28:25.330 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-03T01:28:25.332 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-03T01:28:25.335 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-03T01:28:25.335 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T01:28:25.339 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-03T01:28:25.344 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-03T01:28:25.352 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T01:28:25.353 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.4% 90.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-03T01:28:25.353 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-03T01:28:25.355 INFO:teuthology.orchestra.run.smithi094.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-03T01:28:25.356 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T01:28:25.359 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T01:28:25.361 INFO:teuthology.orchestra.run.smithi094.stderr: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-03T01:28:25.817 INFO:teuthology.orchestra.run.smithi094.stderr: 90.3% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-03T01:28:25.881 INFO:teuthology.orchestra.run.smithi094.stderr: 94.5% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-03T01:28:26.107 INFO:teuthology.orchestra.run.smithi107.stderr: 91.7% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-03T01:28:26.124 INFO:teuthology.orchestra.run.smithi179.stderr: 91.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-03T01:28:26.360 INFO:teuthology.orchestra.run.smithi107.stderr: 94.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-03T01:28:26.412 INFO:teuthology.orchestra.run.smithi094.stderr: 94.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-03T01:28:26.771 INFO:teuthology.orchestra.run.smithi179.stderr: 95.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-03T01:28:27.015 INFO:teuthology.orchestra.run.smithi094.stderr: 90.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-03T01:28:27.644 INFO:teuthology.orchestra.run.smithi107.stderr: 95.2% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-03T01:28:31.826 INFO:teuthology.orchestra.run.smithi179.stderr: 93.7% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-03T01:28:34.389 INFO:teuthology.orchestra.run.smithi094.stderr: 91.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-03T01:28:36.303 INFO:teuthology.orchestra.run.smithi094.stderr: 91.5% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-03T01:28:36.308 INFO:teuthology.orchestra.run.smithi094.stderr: 91.6% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-03T01:28:36.978 INFO:teuthology.orchestra.run.smithi179.stderr: 91.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-03T01:28:37.036 INFO:teuthology.orchestra.run.smithi107.stderr: 91.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-03T01:28:37.703 INFO:teuthology.orchestra.run.smithi094.stderr: 91.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-03T01:28:37.708 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-06-03T01:28:37.708 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m12.419s 2024-06-03T01:28:37.709 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m43.983s 2024-06-03T01:28:37.709 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m3.334s 2024-06-03T01:28:38.254 INFO:teuthology.orchestra.run.smithi107.stderr: 93.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-03T01:28:38.820 INFO:teuthology.orchestra.run.smithi179.stderr: 91.5% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-03T01:28:39.181 INFO:teuthology.orchestra.run.smithi107.stderr: 91.7% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-03T01:28:40.157 INFO:teuthology.orchestra.run.smithi107.stderr: 91.5% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-03T01:28:40.517 INFO:teuthology.orchestra.run.smithi107.stderr: 91.6% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-03T01:28:40.692 INFO:teuthology.orchestra.run.smithi179.stderr: 93.8% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-03T01:28:41.416 INFO:teuthology.orchestra.run.smithi179.stderr: 91.6% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-03T01:28:44.714 INFO:teuthology.orchestra.run.smithi179.stderr: 91.3% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-03T01:28:44.895 INFO:teuthology.orchestra.run.smithi179.stderr: 93.9% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-03T01:28:44.897 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-06-03T01:28:44.897 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m19.625s 2024-06-03T01:28:44.898 INFO:teuthology.orchestra.run.smithi179.stderr:user 1m40.787s 2024-06-03T01:28:44.898 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m2.812s 2024-06-03T01:29:00.850 INFO:teuthology.orchestra.run.smithi107.stderr: 93.4% -- replaced with /var/log/ceph/d037ccd0-2143-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-03T01:29:00.852 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-06-03T01:29:00.852 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m35.580s 2024-06-03T01:29:00.852 INFO:teuthology.orchestra.run.smithi107.stderr:user 1m44.457s 2024-06-03T01:29:00.853 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m3.058s 2024-06-03T01:29:00.853 INFO:tasks.cephadm:Archiving logs... 2024-06-03T01:29:00.853 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi094/log 2024-06-03T01:29:00.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T01:29:07.800 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi107/log 2024-06-03T01:29:07.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T01:29:20.930 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi179/log 2024-06-03T01:29:20.931 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-03T01:29:32.711 INFO:tasks.cephadm:Removing cluster... 2024-06-03T01:29:32.712 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 --force 2024-06-03T01:29:33.037 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T01:29:33.749 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 --force 2024-06-03T01:29:33.941 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T01:29:34.377 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d037ccd0-2143-11ef-bc9b-c7b262605968 --force 2024-06-03T01:29:34.568 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: d037ccd0-2143-11ef-bc9b-c7b262605968 2024-06-03T01:29:34.993 INFO:tasks.cephadm:Removing cephadm ... 2024-06-03T01:29:34.994 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T01:29:35.048 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T01:29:35.063 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-03T01:29:35.078 INFO:tasks.cephadm:Teardown complete 2024-06-03T01:29:35.079 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-03T01:29:35.090 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-03T01:29:35.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T01:29:35.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T01:29:35.105 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-03T01:29:50.879 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-03T01:29:50.892 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-03T01:29:50.892 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T01:29:50.901 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T01:29:50.903 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-03T01:29:50.918 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-06-03T01:29:50.920 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-06-03T01:29:50.923 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T01:29:50.923 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-06-03T01:29:50.923 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +359us[ +361us] +/- 58ms 2024-06-03T01:29:50.924 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -660us[ -658us] +/- 58ms 2024-06-03T01:29:50.924 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +379us[ +381us] +/- 63ms 2024-06-03T01:29:50.924 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:29:50.926 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T01:29:50.926 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-06-03T01:29:50.926 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +357us[ +364us] +/- 58ms 2024-06-03T01:29:50.926 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -645us[ -639us] +/- 58ms 2024-06-03T01:29:50.926 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +367us[ +367us] +/- 63ms 2024-06-03T01:29:50.927 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:29:50.948 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-06-03T01:29:50.959 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-03T01:29:50.959 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-06-03T01:29:50.959 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +337us[ +342us] +/- 58ms 2024-06-03T01:29:50.959 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 104 -641us[ -639us] +/- 57ms 2024-06-03T01:29:50.959 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +383us[ +388us] +/- 63ms 2024-06-03T01:29:50.959 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-03T01:29:50.960 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-03T01:29:50.970 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-03T01:29:50.970 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-03T01:29:50.981 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T01:29:51.196 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T01:29:51.269 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-03T01:29:51.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:29:51.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:29:51.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:29:51.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:29:51.467 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-03T01:29:51.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:29:51.497 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-03T01:29:51.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-03T01:29:51.518 INFO:teuthology.task.internal:Duration was 2630.050952 seconds 2024-06-03T01:29:51.519 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-03T01:29:51.528 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-03T01:29:51.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T01:29:51.536 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T01:29:51.539 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-03T01:29:51.578 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:29:51.580 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:29:51.673 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-03T01:29:52.074 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-03T01:29:52.075 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-06-03T01:29:52.075 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T01:29:52.151 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-06-03T01:29:52.151 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T01:29:52.176 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-06-03T01:29:52.176 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-03T01:29:52.201 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-03T01:29:52.202 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T01:29:52.209 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T01:29:52.218 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-03T01:29:52.291 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-03T01:29:52.291 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T01:29:52.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T01:29:52.299 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-03T01:29:53.046 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-03T01:29:53.058 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-03T01:29:53.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T01:29:53.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T01:29:53.164 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-03T01:29:53.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-03T01:29:53.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:29:53.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:29:53.212 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:29:53.233 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-06-03T01:29:53.254 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-06-03T01:29:53.299 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-06-03T01:29:53.340 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:29:53.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:29:53.388 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:29:53.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:29:53.402 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-03T01:29:53.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-03T01:29:53.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-03T01:29:53.429 INFO:teuthology.task.internal:Transferring archived files... 2024-06-03T01:29:53.430 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi094 2024-06-03T01:29:53.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T01:29:53.540 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi107 2024-06-03T01:29:53.540 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T01:29:53.576 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738748/remote/smithi179 2024-06-03T01:29:53.577 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-03T01:29:53.614 INFO:teuthology.task.internal:Removing archive directory... 2024-06-03T01:29:53.614 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T01:29:53.622 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T01:29:53.624 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-03T01:29:53.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-03T01:29:53.687 INFO:teuthology.task.internal:Not uploading archives. 2024-06-03T01:29:53.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-03T01:29:53.699 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-03T01:29:53.699 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T01:29:53.706 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T01:29:53.708 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-03T01:29:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 01:29 /home/ubuntu/cephtest 2024-06-03T01:29:53.724 INFO:teuthology.orchestra.run.smithi179.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 01:29 /home/ubuntu/cephtest 2024-06-03T01:29:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 3 01:29 /home/ubuntu/cephtest 2024-06-03T01:29:53.756 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-03T01:29:53.768 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-03T01:29:53.865 INFO:teuthology.run:Summary data: client.0-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 2630.05095243454 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi094 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=82e80c0621444e838dfa91c94bbd77ec status: fail success: false 2024-06-03T01:29:53.865 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-03T01:29:53.979 INFO:teuthology.run:FAIL